[REGRESSION]: ArchiveFiles@2 w/ 7z-ultra compress the file in a format that ExtractFiles@1 can't open. #19918

Darth-Shader · 2024-05-28T20:55:41Z

New issue checklist

Task name

ArchiveFilesV2

Breaking task version

231

Last working task version

229

Regression Description

We use heavily the ArchiveFiles@2 and ExtractFiles@1 and we found that since the version 231 ofr ArchiveFiles, the task ExtractFiles@1 is failing with the signature:

ERROR: Unsupported Method :

For binary files like dlls and exe files. Looks like the 7z tool is older now:

I found that if you upgrade to the 2024 version of 7-Zip, the extract works again.

Environment type (Please select at least one enviroment where you face this issue)

Self-Hosted
Microsoft Hosted
VMSS Pool
Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Windows 11

Relevant log output

##[debug]   x
##[debug]   -oC:\Users\HCKTest\AppData\Local\Temp\leyend
##[debug]   C:\root\leyendAutomation\archive.7z
C:\agent\_work\_tasks\ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988\1.229.0\7zip\7z.exe x -oC:\Users\HCKTest\AppData\Local\Temp\leyend C:\root\leyendAutomation\archive.7z
ERROR: Unsupported Method : Bin\ARM64\Release\Automata.dll
ERROR: Unsupported Method : Bin\ARM64\Release\CsvConsumer.dll
ERROR: Unsupported Method : Bin\ARM64\Release\EtwProvider.dll
ERROR: Unsupported Method : Bin\ARM64\Release\GDIProvider.dll
ERROR: Unsupported Method : Bin\ARM64\Release\Leyend_cmd.exe
ERROR: Unsupported Method : Bin\ARM64\Release\Leyend_engine.dll
ERROR: Unsupported Method : Bin\ARM64\Release\msdia140.dll
ERROR: Unsupported Method : Bin\ARM64\Release\msvcdis140.dll
ERROR: Unsupported Method : Bin\ARM64\Release\msvcp140.dll
ERROR: Unsupported Method : Bin\ARM64\Release\msvcp140_atomic_wait.dll
ERROR: Unsupported Method : Bin\ARM64\Release\QProfilerInterface.dll
ERROR: Unsupported Method : Bin\ARM64\Release\QualcommProvider.dll

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 12949856 bytes (13 MiB)

Extracting archive: C:\root\leyendAutomation\archive.7z
--
Path = C:\root\leyendAutomation\archive.7z
Type = 7z
Physical Size = 12949856
Headers Size = 2589
Method = 0A LZMA2:26 LZMA:20 BCJ2
Solid = +
Blocks = 3


Sub items Errors: 18

Archives with Errors: 1

Sub items Errors: 18
ERROR: Unsupported Method : Bin\ARM64\Release\SuperRes_bmp.exe
ERROR: Unsupported Method : Bin\ARM64\Release\vcruntime140.dll
ERROR: Unsupported Method : Bin\x64\Release\msvcdis140.dll
ERROR: Unsupported Method : Bin\x64\Release\msvcp140.dll
ERROR: Unsupported Method : Bin\x64\Release\msvcp140_atomic_wait.dll
ERROR: Unsupported Method : Bin\x64\Release\vcruntime140.dll
##[debug]execResult: {"code":2,"stdout":"\r\n7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21\r\n\r\nScanning the drive for archives:\r\n1 file, 12949856 bytes (13 MiB)\r\n\r\nExtracting archive: C:\\root\\leyendAutomation\\archive.7z\r\n--\r\nPath = C:\\root\\leyendAutomation\\archive.7z\r\nType = 7z\r\nPhysical Size = 12949856\r\nHeaders Size = 2589\r\nMethod = 0A LZMA2:26 LZMA:20 BCJ2\r\nSolid = +\r\nBlocks = 3\r\n\r\n\r\nSub items Errors: 18\r\n\r\nArchives with Errors: 1\r\n\r\nSub items Errors: 18\r\n","stderr":"ERROR: Unsupported Method : Bin\\ARM64\\Release\\Automata.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\CsvConsumer.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\EtwProvider.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\GDIProvider.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\Leyend_cmd.exe\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\Leyend_engine.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\msdia140.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\msvcdis140.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\msvcp140.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\msvcp140_atomic_wait.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\QProfilerInterface.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\QualcommProvider.dll\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\SuperRes_bmp.exe\r\nERROR: Unsupported Method : Bin\\ARM64\\Release\\vcruntime140.dll\r\nERROR: Unsupported Method : Bin\\x64\\Release\\msvcdis140.dll\r\nERROR: Unsupported Method : Bin\\x64\\Release\\msvcp140.dll\r\nERROR: Unsupported Method : Bin\\x64\\Release\\msvcp140_atomic_wait.dll\r\nERROR: Unsupported Method : Bin\\x64\\Release\\vcruntime140.dll\r\n"}
##[debug]Extraction failed for file: C:\root\leyendAutomation\archive.7z 
code: 2 
stdout: 
7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 12949856 bytes (13 MiB)

Extracting archive: C:\root\leyendAutomation\archive.7z

Full task logs with system.debug enabled

UNSUCCESSFUL RUN

 [REPLACE THIS WITH YOUR INFORMATION]

SUCCESSFUL RUN

 [REPLACE THIS WITH YOUR INFORMATION]

Repro steps

No response

wilbit · 2024-05-29T12:15:19Z

This happened because of negligent reviews of #19842.
Team must revert ArchiveFiles task or upgrade ExtractFiles task.

Darth-Shader · 2024-05-29T21:22:47Z

Thanks, make sense. If they can upgrade to the latest 7z for both archiving and extracting, it will be the best solution.

After they fix and merge this issue, do you know how long it takes to replicate the change so I can update my agents?

KonstantinTyukalov · 2024-05-30T11:00:38Z

Hi, thanks for the feedback! We are working on this

yuhungchun · 2024-08-13T15:56:17Z

Do we have an update on this issue?

yuhungchun · 2024-08-17T00:38:05Z

Can anyone confirm the issue is resolved?

ismayilov-ismayil · 2024-08-17T18:45:33Z

Hi @yuhungchun,
currently we have 7z to previous one on ArchiveFiles Task due to issue on Extract files. We are working on to move to new 7z on all tasks (ArcviveFiles and ExtractFiles are not only tasks which uses 7z).
We will update once it released.

yuhungchun · 2024-08-19T03:44:23Z

@ismayilov-ismayil Thank you for the update.

rbgarcia · 2024-08-21T10:53:23Z

Just faced the same issue today (for some reason the pipeline ran successfully until yesterday).
Pretty basic pipeline setup with two jobs. First job archives files using ArchiveFiles@2 and stores them. Second job downloads from store and extracts files using ExtractFiles@1. Clear difference in the version that are being used.

Yesterday (Successful)

Initialize First Job:

Starting: Initialize job
Agent name: 'DVA110030507007'
Agent machine name: 'DVA110030507007'
Current agent version: '2.206.1'
Agent running as: 'Administrator'
Prepare build directory.
Set build variables.
Download all required tasks.
Checking job knob settings.
Finished checking job knob settings.
Start tracking orphan processes.
Finishing: Initialize job

Archive (using task version 2.231.1 and 7-Zip 19.00)

Starting: Archive TestsDir
=====
Task         : Archive files
Description  : Compress files into .7z, .tar.gz, or .zip
Version      : 2.231.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/archive-files
=====
Found 4 files
Archiving file: Directory.Build.props
Archiving file: OutSystems.Spreadsheet.sln
Archiving file: src
Archiving file: tests
C:\agent\_work\_tasks\ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0\2.231.1\7zip\7z.exe a -t7z -mx=1 C:\agent\_work\1\a\TestsDir.7z @C:\agent\_work\_temp\9ukjj6zbycg

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive:
42 folders, 141 files, 73042533 bytes (70 MiB)

Creating archive: C:\agent\_work\1\a\TestsDir.7z

Add new data to archive: 42 folders, 141 files, 73042533 bytes (70 MiB)

Files read from disk: 140
Archive size: 27214618 bytes (26 MiB)
Everything is Ok
Finishing: Archive TestsDir

Initialize Second Job:

Starting: Initialize job
Agent name: 'Azure Pipelines 4'
Agent machine name: 'fv-az629-778'
Current agent version: '3.243.0'
Operating System
Runner Image
Runner Image Provisioner
Current image version: '20240811.1.0'
Agent running as: 'vsts'
Download all required tasks.
Downloading task: DownloadBuildArtifacts (0.242.1)
Downloading task: ExtractFiles (1.231.1)
Downloading task: Npm (1.238.3)
Downloading task: PowerShell (2.242.1)
Downloading task: PublishTestResults (2.242.1)
Checking job knob settings.
Finished checking job knob settings.
Start tracking orphan processes.
Finishing: Initialize job

Extract Files (using task version 1.231.1 and 7-Zip version 16.02)

Starting: Extract Files
====
Task         : Extract files
Description  : Extract a variety of archive and compression files such as .7z, .rar, .tar.gz, and .zip
Version      : 1.231.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/extract-files
====
Found: 1 files to extract:
/home/vsts/work/_temp/TestsDir/TestsDir.7z
Creating destination folder: /home/vsts/work/_temp/Dir
Extracting file: /home/vsts/work/_temp/TestsDir/TestsDir.7z
/usr/bin/7z x -o/home/vsts/work/_temp/Dir /home/vsts/work/_temp/TestsDir/TestsDir.7z

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz (50657),ASM,AES-NI)

Scanning the drive for archives:
1 file, 27214618 bytes (26 MiB)

Extracting archive: /home/vsts/work/_temp/TestsDir/TestsDir.7z
--
Path = /home/vsts/work/_temp/TestsDir/TestsDir.7z
Type = 7z
Physical Size = 27214618
Headers Size = 2530
Method = LZMA2:16 BCJ
Solid = +
Blocks = 6

Everything is Ok

Folders: 42
Files: 141
Size:       73042533
Compressed: 27214618
Finishing: Extract Files

Today (Error during extract)

Initialize First Job

Starting: Initialize job
Agent name: 'DVA110030507007'
Agent machine name: 'DVA110030507007'
Current agent version: '2.206.1'
Agent running as: 'Administrator'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: NuGetAuthenticate (1.244.1)
Downloading task: ArchiveFiles (2.244.1)
Checking job knob settings.
Finished checking job knob settings.
Start tracking orphan processes.
Finishing: Initialize job

Archive (using task version 2.244.1 and 7-Zip version 23.01)

Starting: Archive TestsDir
====
Task         : Archive files
Description  : Compress files into .7z, .tar.gz, or .zip
Version      : 2.244.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/archive-files
====
Found 4 files
Archiving file: Directory.Build.props
Archiving file: OutSystems.Spreadsheet.sln
Archiving file: src
Archiving file: tests
C:\agent\_work\_tasks\ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0\2.244.1\7zip\7z.exe a -t7z -mx=1 C:\agent\_work\1\a\TestsDir.7z @C:\agent\_work\_temp\b6cvj42x30k

7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20

Scanning the drive:
42 folders, 141 files, 73046741 bytes (70 MiB)

Creating archive: C:\agent\_work\1\a\TestsDir.7z

Add new data to archive: 42 folders, 141 files, 73046741 bytes (70 MiB)

Files read from disk: 140
Archive size: 26489352 bytes (26 MiB)
Everything is Ok
Finishing: Archive TestsDir

Initialize Second Job

Starting: Initialize job
Agent name: 'Azure Pipelines 12'
Agent machine name: 'fv-az632-344'
Current agent version: '3.243.0'
Operating System
Runner Image
Runner Image Provisioner
Current image version: '20240811.1.0'
Agent running as: 'vsts'
Download all required tasks.
Downloading task: DownloadBuildArtifacts (0.242.1)
Downloading task: ExtractFiles (1.231.1)
Downloading task: Npm (1.238.3)
Downloading task: PowerShell (2.242.1)
Downloading task: PublishTestResults (2.242.1)
Checking job knob settings.
Finished checking job knob settings.
Start tracking orphan processes.
Finishing: Initialize job

Extract (with error - using task version 1.231.1 and 7-Zip 16.02)
Starting: Extract Files

====
Task         : Extract files
Description  : Extract a variety of archive and compression files such as .7z, .rar, .tar.gz, and .zip
Version      : 1.231.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/extract-files
====
Found: 1 files to extract:
/home/vsts/work/_temp/TestsDir/TestsDir.7z
Creating destination folder: /home/vsts/work/_temp/Dir
Extracting file: /home/vsts/work/_temp/TestsDir/TestsDir.7z
/usr/bin/7z x -o/home/vsts/work/_temp/Dir /home/vsts/work/_temp/TestsDir/TestsDir.7z

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz (50654),ASM,AES-NI)

Scanning the drive for archives:
1 file, 26489352 bytes (26 MiB)

ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libHarfBuzzSharp.dll
ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libSkiaSharp.dll
Error: Extraction failed for file: /home/vsts/work/_temp/TestsDir/TestsDir.7z 
Extracting archive: /home/vsts/work/_temp/TestsDir/TestsDir.7z
code: 2 
stdout: 
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz (50654),ASM,AES-NI)

Scanning the drive for archives:
1 file, 26489352 bytes (26 MiB)

Extracting archive: /home/vsts/work/_temp/TestsDir/TestsDir.7z
--
Path = /home/vsts/work/_temp/TestsDir/TestsDir.7z
Type = 7z
Physical Size = 26489352
Headers Size = 2601
Method = 0A LZMA2:18 BCJ
Solid = +
Blocks = 3


Sub items Errors: 2

Archives with Errors: 1

Sub items Errors: 2
 
stderr: ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libHarfBuzzSharp.dll
ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libSkiaSharp.dll
 
error: undefined;
--
Path = /home/vsts/work/_temp/TestsDir/TestsDir.7z
Type = 7z
Physical Size = 26489352
Headers Size = 2601
Method = 0A LZMA2:18 BCJ
Solid = +
Blocks = 3
Sub items Errors: 2

Archives with Errors: 1

Sub items Errors: 2
##[error]Extraction failed for file: /home/vsts/work/_temp/TestsDir/TestsDir.7z 
code: 2 
stdout: 
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz (50654),ASM,AES-NI)

Scanning the drive for archives:
1 file, 26489352 bytes (26 MiB)

Extracting archive: /home/vsts/work/_temp/TestsDir/TestsDir.7z
--
Path = /home/vsts/work/_temp/TestsDir/TestsDir.7z
Type = 7z
Physical Size = 26489352
Headers Size = 2601
Method = 0A LZMA2:18 BCJ
Solid = +
Blocks = 3


Sub items Errors: 2

Archives with Errors: 1

Sub items Errors: 2
 
stderr: ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libHarfBuzzSharp.dll
ERROR: Unsupported Method : tests/UnitTests/OutSystems.Spreadsheet.Tests/bin/Release/net6.0/runtimes/win-arm64/native/libSkiaSharp.dll
 
error: undefined;
Finishing: Extract Files

rbgarcia · 2024-09-11T18:17:20Z

Even though 7Zip was upgrade to v23 in ExtractFilesV1 task version 1.245.1 in #20339 I'm still getting version 16.02 during the pipeline execution.

Coincidentally, it is the version available by default in the pipeline runner image: https://github.com/actions/runner-images/blob/ubuntu22/20240901.1/images/ubuntu/Ubuntu2204-Readme.md, which I was expecting to be ignored by the task considering there is a specific version being bundled with it.

Looks like the lookup strategy is not consistent between ArchiveFiles and ExtractFiles causing the pipeline to hit the error above.

hunyu · 2024-10-16T21:36:09Z

Do we know if this issue is addressed now? Or people still seeing the same error?

hunyu · 2024-11-05T23:12:04Z

Any update on this Regression issue?

rbgarcia · 2024-11-06T11:31:25Z

The scenario I reported above was probably an edge case, that is not very obvious.
After a while I realized the error only occurred when archiving in a Windows machine and extracting in a Linux one (we are reusing compiled artifacts to also run tests in Linux environments without compiling everything again).

After the update, Archive task would always use 23.03 (in Windows) while the Extract task in Linux would always fallback to 16.02. This would happen when using the 7z format. To workaround the error we basically made the archive type configurable in our templates and started using zip format instead.

hunyu · 2024-11-12T00:02:14Z

Sound like for Extract task under Linux should be updated to use the same version as Windows if 7z format is used?

ismayilov-ismayil · 2024-11-25T10:34:42Z

PR: #20687

Darth-Shader added bug regression This used to work, but a change in the service/tasks broke it. labels May 28, 2024

github-actions bot added Area: Release triage labels May 28, 2024

v-schhabra removed the Area: Release label May 29, 2024

KonstantinTyukalov assigned ismayilov-ismayil May 30, 2024

KonstantinTyukalov added Task: ArchiveFiles and removed triage labels May 30, 2024

ismayilov-ismayil mentioned this issue Aug 22, 2024

Upgrade 7zip on AzurePowershell and other tasks #20339

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[REGRESSION]: ArchiveFiles@2 w/ 7z-ultra compress the file in a format that ExtractFiles@1 can't open. #19918

[REGRESSION]: ArchiveFiles@2 w/ 7z-ultra compress the file in a format that ExtractFiles@1 can't open. #19918

Darth-Shader commented May 28, 2024

wilbit commented May 29, 2024 •

edited

Loading

Darth-Shader commented May 29, 2024

KonstantinTyukalov commented May 30, 2024

yuhungchun commented Aug 13, 2024

yuhungchun commented Aug 17, 2024

ismayilov-ismayil commented Aug 17, 2024

yuhungchun commented Aug 19, 2024

rbgarcia commented Aug 21, 2024

rbgarcia commented Sep 11, 2024

hunyu commented Oct 16, 2024

hunyu commented Nov 5, 2024

rbgarcia commented Nov 6, 2024

hunyu commented Nov 12, 2024 •

edited

Loading

ismayilov-ismayil commented Nov 25, 2024

[REGRESSION]: ArchiveFiles@2 w/ 7z-ultra compress the file in a format that ExtractFiles@1 can't open. #19918

[REGRESSION]: ArchiveFiles@2 w/ 7z-ultra compress the file in a format that ExtractFiles@1 can't open. #19918

Comments

Darth-Shader commented May 28, 2024

New issue checklist

Task name

Breaking task version

Last working task version

Regression Description

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

Azure DevOps Server Version (if applicable)

Operation system

Relevant log output

Full task logs with system.debug enabled

Repro steps

wilbit commented May 29, 2024 • edited Loading

Darth-Shader commented May 29, 2024

KonstantinTyukalov commented May 30, 2024

yuhungchun commented Aug 13, 2024

yuhungchun commented Aug 17, 2024

ismayilov-ismayil commented Aug 17, 2024

yuhungchun commented Aug 19, 2024

rbgarcia commented Aug 21, 2024

Yesterday (Successful)

Today (Error during extract)

rbgarcia commented Sep 11, 2024

hunyu commented Oct 16, 2024

hunyu commented Nov 5, 2024

rbgarcia commented Nov 6, 2024

hunyu commented Nov 12, 2024 • edited Loading

ismayilov-ismayil commented Nov 25, 2024

wilbit commented May 29, 2024 •

edited

Loading

hunyu commented Nov 12, 2024 •

edited

Loading