Azure - AzCopy学习

这篇具有很好参考价值的文章主要介绍了Azure - AzCopy学习。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

使用 AzCopy 将本地数据迁移到云存储空间

  1. azcopy login

    1. 创建存储账号
    ./azcopy login --tenant-id 40242385-c249-4746-95dc-4a0b64d49dc5
    
    • 这里的—tenant-id 在下面的地方查看:目录 ID;需要拥有Storage Blob Data Owner 的权限
    • 账号下可能会有很多目录(tenant),所以在 login 的时候带上 —tenant-id

Azure - AzCopy学习,Azure,azure,学习,Blob

  1. 使用 azcopy copy 上传

    azcopy copy 20230509_144654178_iOS.jpg https://eastorageaccountpg.blob.core.windows.net/private-pg
    

    修改文件名和 endpoint 即可。

如果使用 az 命令也可以打到效果

az storage blob upload -f 20230509_144654178_iOS.jpg -c blog-pg -n 20230509_144654178_iOS.jpg

AzCopy Sync 使用

Azure - AzCopy学习,Azure,azure,学习,Blob

 1. azcopy sync,Client local Dir → EU:Blob/container/dir
    - Client未登录 Azure
    - blob 的 container 生成共享访问令牌,注意权限选择

Azure - AzCopy学习,Azure,azure,学习,Blob

Client 执行 azcopy sync 命令with SAS

        root@ubuntu0:~# azcopy sync "./blog/" "https://eustoragesacc0.blob.core.windows.net/eucontainer?sp=racwdlep&st=2023-07-21T12:27:30Z&se=2023-07-21T20:27:30Z&spr=https&sv=2022-11-02&sr=c&sig=edSES09iSGU1xChuOAJnEKNrTJ1xbk0pR97aO%2FA0boE%3D"
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job e08f5117-f8b7-314d-5aa3-3f22b0fdf82b has started
        Log file is located at: /root/.azcopy/e08f5117-f8b7-314d-5aa3-3f22b0fdf82b.log
        
        100.0 %, 2 Done, 0 Failed, 0 Pending, 2 Total, 2-sec Throughput (Mb/s): 0.6465
        
        Job e08f5117-f8b7-314d-5aa3-3f22b0fdf82b Summary
        Files Scanned at Source: 2
        Files Scanned at Destination: 0
        Elapsed Time (Minutes): 0.0667
        Number of Copy Transfers for Files: 2
        Number of Copy Transfers for Folder Properties: 0 
        Total Number Of Copy Transfers: 2
        Number of Copy Transfers Completed: 2
        Number of Copy Transfers Failed: 0
        Number of Deletions at Destination: 0
        Total Number of Bytes Transferred: 423810
        Total Number of Bytes Enumerated: 423810
        Final Job Status: Completed

查看 Blob 情况

Azure - AzCopy学习,Azure,azure,学习,Blob

  1. EA:blob/container/dir → EU:blob/container/dir

    • ea 源生成 SAS,注意权限
      Azure - AzCopy学习,Azure,azure,学习,Blob

    • Client 执行 azcopy sync 命令with SAS

      root@ubuntu0:~# azcopy sync "https://eastorageacc0.blob.core.windows.net/eacontainer/sync/?sp=rle&st=2023-07-21T12:42:42Z&se=2023-07-21T20:42:42Z&spr=https&sv=2022-11-02&sr=c&sig=%2FZzdp1B5dpZzTGT3PlwhssFKnipWkgp%2BBpOIqo9t1mY%3D" "https://eustoragesacc0.blob.core.windows.net/eucontainer/sync/?sp=racwdlep&st=2023-07-21T12:27:30Z&se=2023-07-21T20:27:30Z&spr=https&sv=2022-11-02&sr=c&sig=edSES09iSGU1xChuOAJnEKNrTJ1xbk0pR97aO%2FA0boE%3D"
      INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
      
      Job 747202e5-1afc-b846-4388-9e2e4eee6c58 has started
      Log file is located at: /root/.azcopy/747202e5-1afc-b846-4388-9e2e4eee6c58.log
      
      100.0 %, 3 Done, 0 Failed, 0 Pending, 3 Total, 2-sec Throughput (Mb/s): 0   
      
      Job 747202e5-1afc-b846-4388-9e2e4eee6c58 Summary
      Files Scanned at Source: 3
      Files Scanned at Destination: 0
      Elapsed Time (Minutes): 0.1
      Number of Copy Transfers for Files: 3
      Number of Copy Transfers for Folder Properties: 0 
      Total Number Of Copy Transfers: 3
      Number of Copy Transfers Completed: 3
      Number of Copy Transfers Failed: 0
      Number of Deletions at Destination: 0
      Total Number of Bytes Transferred: 847620
      Total Number of Bytes Enumerated: 847620
      Final Job Status: Completed
      
    • 查看 Blob 情况

    Azure - AzCopy学习,Azure,azure,学习,Blob

  • AzCopy copy 示例

    azcopy copy "https://eascc20230728.blob.core.windows.net/local/1.txt?sp=racwli&st=2023-07-31T03:54:41Z&se=2023-08-31T11:54:41Z&spr=https&sv=2022-11-02&sr=c&sig=hRxFYgG3chofx1WUwlb0ji5wPuI%2BgA31DxUNZlPjAW4%3D" ./1.txt
    
    • 加上 token 要使用“”
    • https://eascc20230728.blob.core.windows.net/local/1.txt 具体的 blob 还是目录要指定清楚

Blob bw Regions test

Azure - AzCopy学习,Azure,azure,学习,Blob
序号 说明 测试 1 测试 2 测试 3 平均值
1 EA client → SA Container 0.2334 0.2334
2 SA Container → EU client 1.0263 1.2675 1.0148 1.10286667
3 SA Container → EU client @ peer created 1.1209 1.2449 1.1483 1.17136667
4 SA Container → EU Container 4.1681 4.335 4.6351 4.3794
2 Hours Later 1.4338 1.8673 1.65055
5 EU Contaner → SA Container 1.0003 0.9336 0.96695
6 EU client → EU Container 0.767 0.7669 0.767 0.76696667
7 EU Container → EU Client, VM disk iops limited 0.8814 0.8814
8 EA client →EU Container 0.5335 0.7002 0.61685
9 EU container → EA container 0.4001 0.4001

结论:
1. Client 跨Region 获取 Container 中文件,在 Blog 选择微软内网优先的情况下,Region间是否使用 Peer 对等对传输效率基本无影响;
2. Container 的文件直接跨 Region 传输效率与选择的 Region以及时机/方向 有关,一段时间保持稳定;
3. 同 Region 传输效率高;文章来源地址https://www.toymoban.com/news/detail-674420.html

原始数据

    1. EA client → SA Container
        

        root@ubuntu0:/home/zyi# azcopy copy 1.zip https://sastorageac0.blob.core.windows.net/sacontainer
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job f631ebf2-8060-6b47-7164-58044ee0c43c has started
        Log file is located at: /root/.azcopy/f631ebf2-8060-6b47-7164-58044ee0c43c.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                   
        
        Job f631ebf2-8060-6b47-7164-58044ee0c43c summary
        Elapsed Time (Minutes): 0.2334
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed

        
    2. SA Container → EU client
        

        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job bf9d0ff0-6066-2140-4cb9-8716e4272028 has started
        Log file is located at: /root/.azcopy/bf9d0ff0-6066-2140-4cb9-8716e4272028.log
        
        0.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):0.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total,                         2.1 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):6.2 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):11.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)15.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)20.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)24.6 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)29.1 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)33.7 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)38.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)42.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)46.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)51.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)55.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)59.7 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)64.4 %, 0 68.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1710.624 (Disk may be l73.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1845.0545 (77.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1440.6049  82.5 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                                               
        
        Job bf9d0ff0-6066-2140-4cb9-8716e4272028 summary
        Elapsed Time (Minutes): 1.0263
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #download 2
        
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 732e37c9-9e99-364a-4a4e-fa7484fac927 has started
        Log file is located at: /root/.azcopy/732e37c9-9e99-364a-4a4e-fa7484fac927.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                  
        
        Job 732e37c9-9e99-364a-4a4e-fa7484fac927 summary
        Elapsed Time (Minutes): 1.2675
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #download 3
        ^[[Aroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 05c9210a-e503-c046-6f9b-853d6c9a3c4e has started
        Log file is located at: /root/.azcopy/05c9210a-e503-c046-6f9b-853d6c9a3c4e.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                                               
        
        Job 05c9210a-e503-c046-6f9b-853d6c9a3c4e summary
        Elapsed Time (Minutes): 1.0148
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```
        
    3. SA Container → EU client @ peer created
        
        ```powershell
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f has started
        Log file is located at: /root/.azcopy/b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                  
        
        Job b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f summary
        Elapsed Time (Minutes): 1.1209
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #2
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job b646e5c1-af07-6c41-7a0c-641ecfd5d1cd has started
        Log file is located at: /root/.azcopy/b646e5c1-af07-6c41-7a0c-641ecfd5d1cd.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                  
        
        Job b646e5c1-af07-6c41-7a0c-641ecfd5d1cd summary
        Elapsed Time (Minutes): 1.2449
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 30f70c4e-fdd1-9a4b-6e89-c5e7406970de has started
        Log file is located at: /root/.azcopy/30f70c4e-fdd1-9a4b-6e89-c5e7406970de.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                                               
        
        Job 30f70c4e-fdd1-9a4b-6e89-c5e7406970de summary
        Elapsed Time (Minutes): 1.1483
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```
        
    
    1. SA Container → EU Container
        
        ```powershell
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 9168f92f-2da4-6742-6862-8e5696b52a41 has started
        Log file is located at: /root/.azcopy/9168f92f-2da4-6742-6862-8e5696b52a41.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 9168f92f-2da4-6742-6862-8e5696b52a41 summary
        Elapsed Time (Minutes): 4.1681
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/10.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 7cca86e8-11b6-164d-7b33-887d9cf10183 has started
        Log file is located at: /root/.azcopy/7cca86e8-11b6-164d-7b33-887d9cf10183.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 7cca86e8-11b6-164d-7b33-887d9cf10183 summary
        Elapsed Time (Minutes): 4.335
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #3
        root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job c4366598-5822-b946-45b9-6ee9d6f6df15 has started
        Log file is located at: /root/.azcopy/c4366598-5822-b946-45b9-6ee9d6f6df15.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job c4366598-5822-b946-45b9-6ee9d6f6df15 summary
        Elapsed Time (Minutes): 4.6351
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #4
        root@ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 15fe8cca-78c3-3d40-4a55-e90d11495258 has started
        Log file is located at: /root/.azcopy/15fe8cca-78c3-3d40-4a55-e90d11495258.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 15fe8cca-78c3-3d40-4a55-e90d11495258 summary
        Elapsed Time (Minutes): 1.4338
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #5
        root@ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 2ac14c6f-f943-9c4b-7477-6947fd59ab31 has started
        Log file is located at: /root/.azcopy/2ac14c6f-f943-9c4b-7477-6947fd59ab31.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 2ac14c6f-f943-9c4b-7477-6947fd59ab31 summary
        Elapsed Time (Minutes): 1.8673
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```
        
    2. EU Contaner → SA Container
        
        ```powershell
        root@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://sastorageac0.blob.core.windows.net/sacontainer
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 8b6f339f-e23b-2f4e-755c-8bd336942b40 has started
        Log file is located at: /root/.azcopy/8b6f339f-e23b-2f4e-755c-8bd336942b40.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 8b6f339f-e23b-2f4e-755c-8bd336942b40 summary
        Elapsed Time (Minutes): 1.0003
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #2
        root@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://sastorageac0.blob.core.windows.net/sacontainer/1.zip
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 0c1039f0-7a9f-0e44-6a42-079c03708fb8 has started
        Log file is located at: /root/.azcopy/0c1039f0-7a9f-0e44-6a42-079c03708fb8.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 0c1039f0-7a9f-0e44-6a42-079c03708fb8 summary
        Elapsed Time (Minutes): 0.9336
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        ```
        
    
    1. EU client → EU Container
        
        ```powershell
        root@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job d3bdc10a-4142-8b41-7f9a-752c840c5f36 has started
        Log file is located at: /root/.azcopy/d3bdc10a-4142-8b41-7f9a-752c840c5f36.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 623.3303
        
        Job d3bdc10a-4142-8b41-7f9a-752c840c5f36 summary
        Elapsed Time (Minutes): 0.767
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #2
        root@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 3018151c-c777-ca42-4054-7f980de47650 has started
        Log file is located at: /root/.azcopy/3018151c-c777-ca42-4054-7f980de47650.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 598.946 
        
        Job 3018151c-c777-ca42-4054-7f980de47650 summary
        Elapsed Time (Minutes): 0.7669
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #3
        root@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 33cf320d-94a5-aa48-5a35-b94d34cf5cc1 has started
        Log file is located at: /root/.azcopy/33cf320d-94a5-aa48-5a35-b94d34cf5cc1.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 579.6335
        
        Job 33cf320d-94a5-aa48-5a35-b94d34cf5cc1 summary
        Elapsed Time (Minutes): 0.767
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```
        
    2. EU Container → EU Client, VM disk iops limited
        
        ```powershell
        root@eu-ubuntu0:/home/zyi# azcopy copy  https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 10.zip
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 038b5c4a-7376-094d-4e7c-c958e15378f8 has started
        Log file is located at: /root/.azcopy/038b5c4a-7376-094d-4e7c-c958e15378f8.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                                               
        
        Job 038b5c4a-7376-094d-4e7c-c958e15378f8 summary
        Elapsed Time (Minutes): 0.8814
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```
        
    
    1. EA client →EU Container
        
        ```powershell
        root@ubuntu0:/home/zyi# azcopy copy  1.zip https://eustorageac0.blob.core.windows.net/eucontainer
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job e197a453-cd7d-9042-53bc-de6c0e26246c has started
        Log file is located at: /root/.azcopy/e197a453-cd7d-9042-53bc-de6c0e26246c.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                  
        
        Job e197a453-cd7d-9042-53bc-de6c0e26246c summary
        Elapsed Time (Minutes): 0.5335
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        #2
        root@ubuntu0:/home/zyi# azcopy copy  1.zip https://eustorageac0.blob.core.windows.net/eucontainer/10.zip
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 6b1b50a2-3c6c-9c45-5165-085514255e76 has started
        Log file is located at: /root/.azcopy/6b1b50a2-3c6c-9c45-5165-085514255e76.log
        
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total,                                  
        
        Job 6b1b50a2-3c6c-9c45-5165-085514255e76 summary
        Elapsed Time (Minutes): 0.7002
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        
        ```
        
    2. EU container → EA container
        
        ```powershell
        root@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://eastorageaccountpg.blob.core.windows.net/blog-pg
        INFO: Scanning...
        INFO: Autologin not specified.
        INFO: Authenticating to destination using Azure AD
        INFO: Authenticating to source using Azure AD
        INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
        
        Job 04a69131-382f-3741-4263-c81b6e28c27e has started
        Log file is located at: /root/.azcopy/04a69131-382f-3741-4263-c81b6e28c27e.log
        
        INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
        100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 
        
        Job 04a69131-382f-3741-4263-c81b6e28c27e summary
        Elapsed Time (Minutes): 0.4001
        Number of File Transfers: 1
        Number of Folder Property Transfers: 0
        Number of Symlink Transfers: 0
        Total Number of Transfers: 1
        Number of File Transfers Completed: 1
        Number of Folder Transfers Completed: 0
        Number of File Transfers Failed: 0
        Number of Folder Transfers Failed: 0
        Number of File Transfers Skipped: 0
        Number of Folder Transfers Skipped: 0
        TotalBytesTransferred: 10519930170
        Final Job Status: Completed
        ```

到了这里,关于Azure - AzCopy学习的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • Azure Blob MD5算法

    笔者所在项目采用Azure Blob来作为文件存储解决方案, 上传的大文件需要一种机制来验证文件的完整性, 而业内通用的解决方案就是用MD5来做校验. Azure Blob在我们上传文件且没有指定Content MD5时, 就会自动帮我们计算文件的MD5值. 通过Microsoft Azure Storage Explorer工具右键查看任何文件

    2024年01月19日
    浏览(22)
  • [Azure] databricks通过服务主体(service principals)和azure key vaults访问Blob storage

    Service Principal  是Azure Active Directory中的一个应用程序,它被授权访问Azure中的资源。此访问受到分配给服务主体的角色的限制,从而使您能够控制哪些资源可以访问以及在哪个级别上访问,使用专用应用程序服务主体service principals进行本地开发,可以在应用开发期间遵循最低

    2024年04月23日
    浏览(19)
  • Azure Blob 文件下载及本地开发使用配置

    1.我们在本地开发如果没有Azure 账户及blob资源,可以在本地调试开发 所需工具下载:Azure 存储管理器下载可以在本地创建容器Container及Blob 上传所需文件对文件进行操作 也可以管理Azure云的Blob Azure 存储资源管理器 – 云存储管理 | Microsoft Azure 2.打开后创建本地Blob存储  3.本地

    2024年02月07日
    浏览(29)
  • 【Microsoft Azure 的1024种玩法】三十四.将本地数据文件快速迁移到Azure Blob云存储最佳实践

    AzCopy 是一个Azure提供的一款命令行工具,我们可通过简单命令将本地的数据快速复制到 Azure Blob 存储中,本文主要讲述了如何通过AzCopy 工具将本地数据文件快速迁移到Azure Blob云存储 【Microsoft Azure 的1024种玩法】一.一分钟快速上手搭建宝塔管理面板 【Microsoft Azure 的1024种玩法

    2024年02月09日
    浏览(23)
  • 【Azure 存储服务】使用 AppendBlobClient 对象实现对Blob进行追加内容操作

    在Azure Blob的官方示例中,都是对文件进行上传到Blob操作,没有实现对已创建的Blob进行追加的操作。如果想要实现对一个文件的多次追加操作,每一次写入的时候,只传入新的内容?   Azure Storage Blob 有三种类型: Block Blob, Append Blob 和 Page Blob。其中,只有Append Blob类型支持追

    2024年02月03日
    浏览(33)
  • Azure机器学习 - 在 Azure 机器学习中上传、访问和浏览数据

    机器学习项目的开始阶段通常涉及到探索性数据分析 (EDA)、数据预处理(清理、特征工程)以及生成机器学习模型原型来验证假设,本教程介绍如何执行下列操作:将数据上传到云存储空间,创建 Azure 机器学习数据资产,访问笔记本中的数据以进行交互式开发, 创建新版本

    2024年02月06日
    浏览(32)
  • 【Azure 架构师学习笔记】- Azure Private Endpoint

    本文属于【Azure 架构师学习笔记】系列。 公有云的其中一个特点是默认允许公网访问, 这就对企业环境带来风险,也是很多年前企业对公有云抵触的其中一个原因,现在这类问题已经很少,因为有了很多技术来确保云上的资源被安全地访问。其中Private endpoint(PE)就起到了

    2024年03月09日
    浏览(22)
  • Azure - 自动化机器学习AutoML Azure使用详解

    自动化机器学习,简称为AutoML,旨在将机器学习模型的开发中繁琐且重复的任务自动化。这使得数据科学家、分析师以及开发人员能够构建高度可扩展、高效和高性能的ML模型,且不牺牲模型的质量。Azure 机器学习的AutoML功能是基于Microsoft Research团队的前沿技术而开发的。 关

    2024年02月08日
    浏览(24)
  • 【Azure 架构师学习笔记】- Azure Databricks (9) -- UC权限

    本文属于【Azure 架构师学习笔记】系列。 本文属于【Azure Databricks】系列。 接上文 【Azure 架构师学习笔记】- Azure Databricks (8) --UC架构简介 UC 是Databricks进行数据治理,集中权限管控, 提高大规模数据存储,共享安全性的新工具。通过基于角色的访问控制(Role-based access contr

    2024年02月22日
    浏览(19)
  • 【Azure 架构师学习笔记】- Azure Databricks (8) --UC架构简介

    本文属于【Azure 架构师学习笔记】系列。 本文属于【Azure Databricks】系列。 接上文 【Azure 架构师学习笔记】- Azure Databricks (7) --Unity Catalog(UC) 基本概念和组件 UC 简单来说,就是管理两样东西:用户和元存储。 所有Databricks的用户和Service Principal都存储在UC内部。这些实体的授权

    2024年02月21日
    浏览(21)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包