Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finetuning for Improved Small Icon Detection in OmniParser #3

Open
abrichr opened this issue Nov 2, 2024 · 0 comments
Open

Finetuning for Improved Small Icon Detection in OmniParser #3

abrichr opened this issue Nov 2, 2024 · 0 comments

Comments

@abrichr
Copy link
Member

abrichr commented Nov 2, 2024

  1. Objective:

    • Implement fine-tuning for OmniParser’s YOLO model to enhance detection accuracy on small icons and UI elements.
  2. Context:

    • Current limitations in detecting small or densely packed icons due to model sensitivity thresholds.
  3. Proposed Solution:

    • Data Collection: Assemble a labeled dataset of small icons/UI elements, including bounding boxes in YOLO format.
    • Training Configuration: Use YOLO-specific parameters, adjusting image size (e.g., 640x640) and hyperparameters to improve small object sensitivity.
    • Integration Steps:
      • Modify get_yolo_model to support loading the fine-tuned model.
      • Update config to reference the fine-tuned model.
      • Provide a train_yolo function to manage the fine-tuning process.
    • Testing: Evaluate detection accuracy on new test images containing small icons/UI elements, adjusting BOX_THRESHOLD as needed.
  4. Expected Outcome:

    • More accurate small icon detection, fewer missed icons in dense layouts, and reduced reliance on preprocessing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant