Uploading Large Files into Amazon S3

I recently had to upload some VHDs to Amazon S3 and found myself going beyond the upload limits for the web upload.  In order to accomplish uploading the 50GB files I used the AWS Command Line Interface (CLI).  If you need to know how to install the CLI check here (Install CLI)

1. Open a command prompt As Administrator.

2. Make sure your CLI configuration is up to date.

C:\Windows\System32> aws configure
AWS Access Key ID: yourkey
AWS Secret Access Key: youraccesskey
Default region name [us-east-1]: yourregion
Default output format [None]: json

3. Create your bucket on S3 if it's not already there.

4. Run the following command.  

C:\Windows\System32> aws s3 cp c:\temp\yourfile.vhd s3://yourbucket/VHDs

5. After you run the command the output window will provide you feedback as to how much of your download is complete and your current upload speed. 

C:\Windows\System32> aws s3 cp c:\temp\yourfile.vhd s3://yourbucket/VHDs
Completed 1.1 GiB/50 Gib (14.1 MiB/s) with 1 file(s) remaining

Comments

Post a Comment

Popular posts from this blog

Add User As Local Administrator On Domain Controller

I recently was settting up a new Microsoft SharePoint 2010 machine and had promoted the machine to a domain controller before creating my SharePoint admin accounts.  I needed to add several of my accounts to the local Administrators group.  Unfortunately after you promote a server to a domain controller you can no longer access the GUI for Local Users and Groups.  Instead I had to use the command line to add the users. Open a command promt using the "Run as administrator" function and then run the following command. net localgroup Administrators /add {domain}\{user} Note: do not include the {} brackets.
Read more

How to Create SharePoint Items with Power Automate Desktop

Image
Overview Power Automate Desktop is a great way to automate many of your daily task so you can focus on real work. A prime example of this is getting data from one place to another, especially when those data sources do not have an API such as a legacy desktop application or a file. In this example I demonstrate several ways in which an Excel sheet containing fictitious customer data could be loaded into a SharePoint list. In order to do this I first generated some fake data for my customers using Mockaroo . I then created a new SharePoint list and added some columns to match my spreadsheet. Methods My original plan was to just use the PAD recorder , which I did, but after creating that Flow I decided to find other possible ways to accomplish the task. The list below is not a complete list but should provide some ideas. The flexibility of PAD allows for an even wider range of possibilities for carrying out tasks such as these. At the end of each section I have placed the s...
Read more

Install/Configure Power Apps Mobile for U.S. Government and DoD User

Image
For those working in the U.S. Government or DoD/Military space needing the Power Apps mobile application on your phone, an additional step of selecting a region must be taken before you attempt to sign-in. Microsoft has three separate regions listed below where U.S. Government information is stored, for additional information click here . US Government GCC US Government CGG High US Government DoD NOTE: Before attempting the video or written instructions below, contact your system administrator to determine which U.S. Government region you belong to. Video Tutorial Click the image below to open the YouTube video. Instructions Here are the instruction on how to install the Power Apps mobile application and change the region setting before you log in. Android : Open Google Play / iPhone : Open App Store Search for “powerapps” Install the Power Apps mobile app Open the Power Apps mobile app Click the gear icon in the lower right hand corner of the app. Select “Region se...
Read more