testSetDsetFilesStatus.1

Add change dbs dataset file status fix 5204 (#5243)

* add setdataset.py for #5204

* refactor and add setfiles

* add Content-type arg to HTTPRequests

* setdataset to use contentType

* rename commands to setdatasetstatus setfilestatus

* add autocomplete

* list of LFNs not supported yet

* some pylint and pep8

* add logging for setfilestatus

py3cmssw10.1

Use python3 and rucio client with cmssw 10 (#5237)

* use Python3 for CMSSW_10, add our LumiList, fix LumiList import, remove
pycurl from crab.py

* fix PSetCores check in CMSSWConfig.py

* remove unused, obsolete code from report.py

v3.231010

small fixes to clean list of pending top/high-prio issues
before tackling the enhancement which require integration with Ruciio Client

List of commits:

• Reduce curl verbosity (#5218)
• limit list of sandbox content to console to 5 files. Fix #5200 (#5217)
• Update comments in WMCoreConfiguration.py

Validation
I have sent ClientValidation Jenking tests for v3.231010 and all CMSSW releases in our list
dmwm/CRABServer#7908 - OK
dmwm/CRABServer#7909 - OK
dmwm/CRABServer#7910 - OK
dmwm/CRABServer#7911 - OK
dmwm/CRABServer#7912 - OK
dmwm/CRABServer#7913 - OK
dmwm/CRABServer#7914 - OK
dmwm/CRABServer#7915 - OK
dmwm/CRABServer#7916 - OK

Deployment
set crab-dev to this: cms-sw/cmsdist#8746
As of Oct 12, available as crab-dev on IB

v3.231009

a small, safe, tested update on last release to have a "final tag" before working on adding Rucio client
List of changes:

• always put bin and python dirs in sanbox. #5215 - fixes #5210 and #5216

v3.230925

Small improvement and bug fix.

• Print rule id instead of container name in "crab status" command by @novicecpp in #5205
• Print Rucio info for Rucio ASO task (updated version of #5205) by @novicecpp in #5209
• skip .git directories when creating sandbox. Fix #5202 by @belforte in #5207
• change getLumiListInValidFiles to use dasgoclient Fix #5175 by @belforte in #5211

Full Changelog: v3.230612...v3.230925

v3.230612

Minor improvements to make ti easier to run on other architectures, as requested by Kevin Pedro

add conf/SCRAM/hooks (all of the times) and venv (on request) subdirs of CMSSW_BASE to sandbox.
Solves #5196 and #5197 via #5199

v3.230417

Brings just one important check to prevent requesting >1 cores and only using 1

• Make sure numberOfThreads is uint32 in the PSet #5194 (#5195)

v3.230404

add crab checkdataset command to check if a dataset is available on disk for processing via CRAB

Client validation after deployment done via dmwm/CRABServer#7584

v3.230227

This tag adds two main things:

1. support for Data.inputBlocks
   • new Data.inputBlocks parameter by @novicecpp in #5185
   • Fix getattr with config.Data.inputBlocks by @novicecpp in #5186
2. An improvement on which files CRABClient automatically adds to the usertarball, that makes it compatible with newest cmssw versions
   • add src/**/python to user sandbox by @mapellidario in #5188

Full Changelog: v3.230124...v3.230327

v3.230124

The main motivation for this new CRABClient release is to make crab client commands submit --dryrun and preparelocal compatible with the new JobWrapper introduced by https://github.com/dmwm/CRABServer/releases/tag/v3.221128.patch1 .

Moreover, it will bring in a new argument for crab configuration file that allows to specify the accelerator parameters, by Wa.

There are also a couple of "cosmetic" changes to what crab commands print to stdout.

What's Changed

• adapt CRABClient to new jobwrapper by @mapellidario in #5180
• new Site.acceleratorParams by @novicecpp in #5179
• do not write unnecessary msg to user by @belforte in #5181
• submit --dryrun. print info about last trial by @mapellidario in #5184

Full Changelog: v3.221018...v3.230124