Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Info extractor rework #890

Open
dyc3 opened this issue May 15, 2023 · 0 comments
Open

Info extractor rework #890

dyc3 opened this issue May 15, 2023 · 0 comments
Labels
help wanted Extra attention is needed refactor Something needs to be reworked to improve code quality server Improvements or additions to the server

Comments

@dyc3
Copy link
Owner

dyc3 commented May 15, 2023

The info extractor has some problems that make implementing new features hard.

  • URL parsing is too tightly coupled to metadata extraction.
  • There isn't a clean way to obtain the URL that was resolved into a Video or array of Video
  • Collections are not as clearly defined as Video is, resulting in lots of duplicate code.
  • ServiceAdapters don't have a clean, easy to follow mechanism to provide Videos where service != adapter.serviceId
  • There isn't a clean, super type-checked way to provide startAt and endAt metadata that is embedded into the url (eg. youtube's t=5s query parameter for starting the video at 5 seconds)
  • "highlighting" a video in an add preview is incredibly janky, (and i think its broken right now)
    • highlighting is a feature that shows the referenced video at the very top of the list if you put in a youtube link with a playlist and a video referenced (eg. https://www.youtube.com/watch?v=OtqD0ddbLic&list=PLqKpXXbHNvC83j0C8UsR817qcrXupLGZ5)

I want the new info extractor to be able to:

  • resolve all the above issues
  • do pagination (Add Preview Pagination #172)
  • provide extra metadata for queue items (eg startAt and endAt)
  • maintain all the caching and bandwidth saving logic we already have
    • maybe make it possible to cache add previews locally
  • be able to cache collections (playlists, channels, etc.)

So far, what I'm thinking, when diagrammed out, is this:

classDiagram

    class VideoId {
        service: string
        id: string
    }

    class Video {
        id: VideoId
        meta: VideoMetadata
    }

    Video <-- VideoId

    class CollectionId {
        service: string
        id: string
        page: string
    }

    class ExtractStrategy {
        <<abstract>>
        resolve(url) Video[]
    }

    ExtractStrategy <|-- Single
    ExtractStrategy <|-- Multi
    Multi <|-- Collection

    class ServiceAdapter {
        <<abstract>>
        canHandleUrl(url) boolean
        isCollectionUrl(url) boolean
        parseSingle(url) VideoId
        fetchSingle(videoId) Video
        fetchMulti(videoId[]) Video[]
        parseCollection(url) CollectionId
        fetchCollection(collectionId) VideoId[]
    }

    Youtube <|-- ServiceAdapter
    Vimeo <|-- ServiceAdapter
    Direct <|-- ServiceAdapter
Loading

This isn't perfect or complete, and I'll be refining it and updating this issue.

Some notes:

  • In CollectionId, page is a string so that it can either be a number or a page token (youtube api uses those)
  • ServiceAdapter should provide a default, overrideable impl for fetchMulti(), allowing derived adapters to override it with something more optimized if applicable.
@dyc3 dyc3 added help wanted Extra attention is needed server Improvements or additions to the server refactor Something needs to be reworked to improve code quality labels May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed refactor Something needs to be reworked to improve code quality server Improvements or additions to the server
Projects
None yet
Development

No branches or pull requests

1 participant