v3.240520

this release allows to use userInutFiles configuration parameter as a "fileMask" to select which files to process from the input dataset.

Requires REST and TW v3.240517 or later

• 240612 REST and TW are OK now, move to prod via cms-sw/cmsdist#9242

What's Changed

• check gfal-copy exit status first. Fix #5310 by @belforte in #5311
• use None as default for optional string config param. Fix #5309 by @belforte in #5312
• allow inputDataset and userInputFiles at same time. Fix #5308 by @belforte in #5313

Full Changelog: v3.240416...v3.240520

v3.240416

description

add two minor enhancements
the second one is mandatory in order to be able to use a Robot certificate for myproxy retrieval in TaskWorker
w/o forcing users to create new myproxy crdential at each crab command

• allow : in DN of myproxy retrievers. Fix #5303 (#5304)
• do not crash on Rucio unavailable. Fix #5298 (#5302)

validation

• deployed in IB as crab-dev with cms-sw/cmsdist#9138
• tested (OK) with Jenkins CV and CCV
  • dmwm/CRABServer#8343
  • dmwm/CRABServer#8344
  • dmwm/CRABServer#8345

deployment

• deployed in IB as crab-prod with cms-sw/cmsdist#9147

v3.240404

adds fix for #5301 which is needed for CMSSW_13 and later

v3.240216

This release introduce a break change on crab preparelocal command. Please see .

What's Changed

• properly report invalid run_range in config by @belforte in #5286
• preparelocal creates one json file with args for job. Fix #5276 by @belforte in #5287
• Fix checkfile by @belforte in #5292
• avoid python stuff in .local Fix #5290 by @belforte in #5293

Full Changelog: v3.240110...asdfadsfsadf

v3.240110

Main purpose of this release is to fix usage of python3.
Until now we only checked that "python3 is available so we add Rucio client".
But submit command is special, it requires manipulation of PSet.py so
it must be run with same python version as used internally by cmsRun.

Validation of this release is described in dmwm/CRABServer#8147

The changes for that fix these two issues

• allow users to force submit with python2 #5239
• client v3.231101 creates bad PSet.pkl #5279

and other problems similar to last one as reported by users in following cms-talk threads

• https://cms-talk.web.cern.ch/t/crab-job-failed-with-exit-code-7002/32986
• https://cms-talk.web.cern.ch/t/issue-with-crab-submit/33142/5
• https://cms-talk.web.cern.ch/t/issue-with-crab-submit/33039

I have verified using CMSSW_10_2_6 and CMSSW_10_6_30 that current crab-prod (v3.231101) reproduces all those problems while
they are gone with crab client head (i.e. this tag).

Other minor fixes with respect to previous release v3.231204 are:

• fix checkfile reporting #5280 (#5284)
• correct bytes-to-GB conversion in checkdataset fix #5282 (#5283)
• create proper classAd file for local use (#5277)
• Fix help for crab resubmit. fix #5273 (#5275)

Full comparison with latest release v3.231204 v3.231204...v3.240110

Full comparison with latest crab-prod version v3.231101 v3.231101...v3.240110

v3.231204

add 1 commit to v3.231201

• fix and improve checkdataset : #5263 #5272

v3.231201

Brings three main changes:

• every command that requires a --dir option, can now be executed also with --task and --instance as well
  • all crab commands that require a projdir can be called with --task= by @mapellidario in #5269
  • bash completions - every command that uses --dir also supports --task/--instance by @mapellidario in #5270
• crab recover - new command by
  • @mapellidario in #5250
• all crab client commands support py2 and py3. we use py3 from cmssw_10_2 or later
  • make all file compile on py2/3 fix #5261 by @belforte in #5262
  • only use python3 for CMSSW_10_2 or later. Fix #5264 by @belforte in #5265
  • proper way to use py3 from CMSSW_10_2 onward. Fix #5264 by @belforte in #5266

other changes

• trim docstring for class to look good on --help by @belforte in #5249
• add deprecation reason for sendPythonFolder by @belforte in #5251
• add checkfile.py fix #5234 by @belforte in #5252
• Use isEnoughRucioQuota from ServerUtilities.py by @novicecpp in #5247
• Fix #5253 by @belforte in #5254
• fix bug in #5254. Why didn't pylint spot this ? by @belforte in #5256
• Cleanup crab bash completion 5255 by @belforte in #5259
• Move checkdataset to crab client repo fix 5233 by @belforte in #5260

Full Changelog: v3.231030...v3.231201

v3.231101

simply a renaming of v3.231030.p1 to stick to standard v3.YYMMDD schema and better fit into cmsdist automatic machinere. I picked Nov 01 simply to allow possibility for 02, 03... if needed, besides being almost the day after 231030 which is being patched in here
Content:

• proper way to use py3 from CMSSW_10_2 onward. Fix #5264 (#5267)

Lit of changes was in v3.231010...v3.231030.p1

Now will be v3.231010...v3.231101

v3.231030

Brings 2 big changes:

1. integration with Rucio python client and Rucio-aware checkwrite and status
2. crab setdatasetstatus and crab setfilestatus

Detailed list of changes: v3.231010...v3.231030

testSetDsetFilesStatus.2

Test setdatasetfilesstatus (#5246)

* add setdataset.py for #5204

* refactor and add setfiles

* add Content-type arg to HTTPRequests

* setdataset to use contentType

* rename commands to setdatasetstatus setfilestatus

* add autocomplete

* list of LFNs not supported yet

* some pylint and pep8

* add logging for setfilestatus

* more HTTPRequest,CRABRest,getDBSRest to new RestInterfaces.py

* do not pass version to REST clients, it is set in HTTPRequests

* removed one import __version__ too many !

* fix use of version and UserAgent

* fix use of version and UserAgent

* simpley make userAgent=CRABClient/__version__ the default

* cleanup use of Content-type

* cleanup use of Content-type

* add comment