Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return "None" when dspy.Image is missing for the input, similarly to str input fields #1913

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rogeriochaves
Copy link
Contributor

@rogeriochaves rogeriochaves commented Dec 9, 2024

when using the new dspy.Image field, it's not possible to make it optional (for example if you have 2-3 images as input, but not all of them are required). For str input fields, that's not a problem, as None inputs are just rendered as a literal "None" on the prompt for the LMs

simple example, this breaks for the image field, but not for the description field:

class ImageReader(dspy.Signature):
    """What is in the image?"""

    image: dspy.Image = dspy.InputField()
    description: str = dspy.InputField()
    response: str = dspy.OutputField()


dspy.Predict(ImageReader)(image=None, description=None)

I propose then simply returning "None", explicitly as a text type, so the LM knows clearly that the input is lacking an image

@okhat okhat requested a review from isaacbmiller December 9, 2024 17:52
@okhat
Copy link
Collaborator

okhat commented Dec 10, 2024

Hey @rogeriochaves !! Can one do typing.Optional[dspy.Image]? I'm guessing that's not supported right now, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants