Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Start modelling the standard library using MaD #15905

Conversation

yoff
Copy link
Contributor

@yoff yoff commented Mar 13, 2024

I am thinking to add a collective change note at the end mentioning "several standard library functions" or some such. Let me know if you want incremental change notes instead..

- empty models for now
- `summaryModel` of `codeql/python-all` will be added to shortly.
@yoff yoff requested a review from a team as a code owner March 13, 2024 14:46
@yoff yoff marked this pull request as draft March 13, 2024 14:57
- `quote` together with `re.compile` recover regex injection alerts on haiwen/seahub
- `quote_plus` recovers the URL redirection alert on DemocracyClub/EveryElection
- `unquote` recovers path injection alerts on `cloudera/hue`
- it was tedious finding justifications for the rest..
@yoff yoff force-pushed the python/stdlib-start-modeling-using-mad branch from bf4f2cb to e02e4e8 Compare March 13, 2024 14:59
yoff added 2 commits March 14, 2024 09:22
There is already a model there so we add to that one.

We did observe that this existing model was blocked by the external MaD model.
This is concerning and needs to be cleared up.
@yoff yoff added the no-change-note-required This PR does not need a change note label Mar 14, 2024
@yoff yoff marked this pull request as ready for review March 14, 2024 09:46
@RasmusWL
Copy link
Member

I am thinking to add a collective change note at the end mentioning "several standard library functions" or some such. Let me know if you want incremental change notes instead..

sounds fine 👍

Copy link
Member

@RasmusWL RasmusWL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you highlight why you put the models where you did? that is, why not put them under python/ql/lib/semmle/python/frameworks/?

Also, did you consider whether quote/qoute_plus should be sanitizers for any queries? (I can't think of any myself 🤔)

otherwise LGTM

Comment on lines 21 to 22
- ["urllib", "Member[parse].Member[splitquery]", "Argument[0,url:]", "ReturnValue.TupleElement[0]", "taint"]
- ["urllib", "Member[parse].Member[splitquery]", "Argument[0,url:]", "ReturnValue.TupleElement[1]", "taint"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if you could do "ReturnValue.TupleElement[0,1]"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, maybe we already can. I should add a test... :-)

@yoff
Copy link
Contributor Author

yoff commented Apr 9, 2024

Can you highlight why you put the models where you did? that is, why not put them under python/ql/lib/semmle/python/frameworks/?

I followed C# (which has the same structure as Java and Go) but that was just because I did not realize that we already have a natural place for it...I will move it :-)

Also, did you consider whether quote/qoute_plus should be sanitizers for any queries? (I can't think of any myself 🤔)

Yes, but I did not immediately see any. We have shlex.quote as a sanitizer for UsafeShellCommandConstruction but that is quite specialised to shells. It did make me wonder what takes precedence, a MaD model or a sanitizer. I believe a sanitizer will, but I will check..

@yoff yoff requested a review from RasmusWL April 9, 2024 11:33
@yoff yoff requested review from a team as code owners April 18, 2024 09:40
@yoff yoff requested a review from a team April 18, 2024 09:40
@yoff yoff requested review from a team as code owners April 18, 2024 09:40
@github-actions github-actions bot added the C# label Apr 18, 2024
@RasmusWL RasmusWL marked this pull request as draft April 18, 2024 09:43
@RasmusWL RasmusWL removed request for a team April 18, 2024 09:43
@RasmusWL
Copy link
Member

Sorry for the ping, that's my fault 😬

@yoff yoff marked this pull request as ready for review April 24, 2024 09:05
@yoff yoff merged commit 2357a49 into github:yoff-python-stop-extracting-std-lib Apr 24, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-change-note-required This PR does not need a change note Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants