Release 080 (#549)

* Release 080 (#493) * Start SQLAlchemy intgr * RE: #2 .. Save Work * RE: #2 sync changes * re: #2 end of day sync * Re: #2 , Finish program outline and replace core components. * RE: #2 sync * Re: #2 still some known issues * Re: #2 - Running Alpha - V1 * Re: #2 Remove final known item_id.parent_id * Re: #2 Something's happening! * RE: #3 alpha-2 * Re: #3 final sync all changes * Re: #2 Fix failed symlinker to not always immediately yield the item back --------- Co-authored-by: Administrator <[email protected]> * fix: tidy. removed clean_title. * fix(deps): replace psycopg2 with psycopg2-binary * chore: added libpq-dev dep to dockerfile * Various fixes (#509) * Fix: Actually add new items to the database and back to queue * Fix: Pass existing item to state_transition if one exists * feat: add critical error message and fail out when trying to enhance the same item due to multiple folder entries * Fix: add the item_id to the added array so the enhancement error will trigger. * Fix: Don't expunge the objects --------- Co-authored-by: Administrator <[email protected]> * Fix: Ensure streams object is recreated when loading from database and set to a default value and remove the static instance from the class definition. (#513) Co-authored-by: Administrator <[email protected]> * Release 080 patches (#515) * Fix: Clarify log message about backoff timer being hit from the scraper. * Tidy: Remove pre-loading of children and setting their parents Fix: Always set the streams from the old input item on calls to _get_item_from_db Fix: Remove redundant copy streams Feat: Add HARD_RESET env variable to erase the database on start * Tidy: Remove always print HARD_RESET value --------- Co-authored-by: Administrator <[email protected]> * Comet Scraper (#511) * Add first version of comet scraper Fix annatar not using custom url for validation * Fix imdb id in comet scraper * Finish comet scraper * feat: improved ui (#422) (#512) * testing * testing * minor fixes * feat: some frontend ui changes :) * minor tweaks * added movies and top movies section * nearly completed homepage * fix: fix settings and improvements to homepage * chore: update app name to Riven and fix page header in library and onboarding * feat: switch to vaul-svelte dependency for mobile ui and improvements to ui in general * chore: update app name to Riven, improve UI consistency and changes to items endpoint * feat: Add incomplete items to statistics page * feat: Add services status to statistics page feat: add lazy loading for images in statistics and home pages (#502) fix: min/max filesize being returned undefined fix: minor ui improvements (#503) * fix: minor ui improvements * chore: formatted files fix: lower the z index and increase z index of header (#504) feat: add top rated section (#505) * feat: add top rated section * chore: format files fix: text color on light theme (#506) chore(deps-dev): bump vitest from 1.6.0 to 2.0.1 in /frontend (#498) Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 1.6.0 to 2.0.1. - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Commits](https://github.com/vitest-dev/vitest/commits/v2.0.1/packages/vitest) --- updated-dependencies: - dependency-name: vitest dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Rate limiting for RD - Change requests models to take through an optional instance of the rate limiter class - Implement rate limiter settings for RD, a Conservative 100 calls per 60 seconds as default - Ensure if enabled the rate limiter is initialised within the RD class - Ensure all Rd calls (apart from ping) use the initialised rate limiter if present in requests Add RD rate limit settings to general settings pane - frontend Ignore mac and rider specific settings remove config from frontend - this is internal now, non configurable remove config, and use oll the time 1 per second Split out rate limiter to seprate file, also work in 429's and handle overall rate limiting and endpoint specific ratelimiting Use get for ping, and pass rate limiters Added a user agent factory, and randomiser in requests Updated zilean scraper to filtered endpoint Only on movies. Aired at wont be right for seasons, we need a better way to do this across the board * fix: add missing os import * Feature/alldebrid support (#517) * All Debrid Support (backend) * Frontend support for alldebrid settings * Refactor Zilean scraper for MediaItem support (#516) The Zilean scraper has been refactored to support the MediaItem class. Previously, it only supported the Movie class. The 'aired_at' attribute of a Movie instance is now replaced with the 'year' attribute of a MediaItem instance. This change allows more flexibility in handling different types of media items beyond just movies. * fix: add check for Overseerr pageInfo attr on results * fix: check results length appropriately on overseerr * fix: added check for `None` filenames during symlinking * fix: tidy initialize on fresh start * fix: downloader init validation and run fixed. none.mkv fixed from symlinking * Settings migration (#527) * Settings migration Migrate settings from older versions to new version on run. Saves settings after model assignment * Ensure db config is written * DB Migrations Path - use utils data dir. (#533) * Settings migration Migrate settings from older versions to new version on run. Saves settings after model assignment * Ensure db config is written * hotfix db migration path * fix: fix wanted shows in rd. fixed /stats endpoint * fix: temporarily disable blacklisting from RD * Fix yield of plex watchlist (#538) * fix: fixed bug with /services endpoint with downloaders init * fix: bandaid frontend endpoints for now * chore!: renamed backend dir to src. updated Dockerfile * fix: out with old :) * fix: update Dockerfile.slim to use src dir * fix: update entrypoint to src dir * fix: update Dockerfile to use src dir * fix: update workflow and makefile, and coverage * tidy: add logging to rd. fix items controller. * Fixes: better fill in enchanced mediaitems from symlinks and modify program to use id for queue checking (#541) * Fix: media-enhancer creates blank items for missing seasons and episodes * Fix: Check for items in queue by _id instead of comparing objects to prevent issues. * Fix: Remove erronous returns * Feat: Try and scrape entire show/season again if backoff timer allows when in partially completed * Fix: Allow shows and seasons to return scraped state if they're scraped so they can proceed to the next phase --------- Co-authored-by: Administrator <[email protected]> * feat/Torbox cached downloader support * fix: removed cache entirely. loosened up rd packs to accept any episode that matches instead of requiring full packs * fix: plex watchlist has been fixed * Remove frontend * Remove frontend from docker (#534) --------- Co-authored-by: Matthew Adams <[email protected]> Co-authored-by: Administrator <[email protected]> Co-authored-by: Dreu LaVelle <[email protected]> Co-authored-by: Davide Marcoli <[email protected]> Co-authored-by: Gaisberg <None>
rivenmedia · Jul 22, 2024 · b6e6bc3 · b6e6bc3
1 parent 0355e64
commit b6e6bc3
Show file tree

Hide file tree

Showing 330 changed files with 3,182 additions and 14,240 deletions.
diff --git a/.coveragerc b/.coveragerc
@@ -1,6 +1,6 @@
 [run]
 branch = True
-source = backend
+source = src
 
 [report]
 omit =

diff --git a/.github/workflows/backend-battery.yml b/.github/workflows/backend-battery.yml
@@ -34,14 +34,14 @@ jobs:
 
     # - name: Ruff & Isort Check
     #   run: |
-    #     poetry run ruff check ./backend
-    #     poetry run isort --check-only ./backend
+    #     poetry run ruff check ./src
+    #     poetry run isort --check-only ./src
 
     # - name: Type check
     #   run: poetry run pyright
 
     # - name: Run Tests & Coverage
-    #   run: poetry run pytest --cov=./backend --cov-report=xml
+    #   run: poetry run pytest --cov=./src --cov-report=xml
 
     # - name: Upload Coverage Report to Codecov
     #   uses: codecov/[email protected]

diff --git a/.gitignore b/.gitignore
@@ -49,4 +49,10 @@ env/
 venv/
 ENV/
 env.bak/
-venv.bak/
+venv.bak/
+
+# Rider IDE
+**/.idea/
+
+# MacOs
+**/.DS_Store
diff --git a/Dockerfile b/Dockerfile
@@ -24,14 +24,6 @@ COPY pyproject.toml poetry.lock ./
 RUN touch README.md
 RUN poetry install --without dev --no-root && rm -rf $POETRY_CACHE_DIR
 
-# Frontend Builder
-FROM node:20-alpine AS frontend
-WORKDIR /app
-COPY frontend/package*.json ./
-RUN npm install -g pnpm && pnpm install
-COPY frontend/ .
-RUN pnpm run build && pnpm prune --prod
-
 # Final Image
 FROM python:3.11-alpine
 LABEL name="Riven" \
@@ -42,59 +34,36 @@ LABEL name="Riven" \
 ENV PYTHONUNBUFFERED=1
 RUN apk add --no-cache \
     curl \
-    fish \
     shadow \
-    nodejs \
-    npm \
     rclone \
-    fontconfig \
     unzip \
     gcc \
     musl-dev \
     libffi-dev \
-    python3-dev && \
-    npm install -g pnpm
-
-# Install Nerd Fonts
-RUN mkdir -p /usr/share/fonts/nerd-fonts && \
-    curl -fLo "/usr/share/fonts/nerd-fonts/FiraCode.zip" \
-    https://github.com/ryanoasis/nerd-fonts/releases/download/v2.1.0/FiraCode.zip && \
-    unzip /usr/share/fonts/nerd-fonts/FiraCode.zip -d /usr/share/fonts/nerd-fonts && \
-    rm /usr/share/fonts/nerd-fonts/FiraCode.zip && \
-    fc-cache -fv
+    python3-dev \
+    libpq-dev
 
 # Install Poetry
 RUN pip install poetry==1.8.3
 
-# Create fish config directory
-RUN mkdir -p /home/riven/.config/fish
-
 # Set environment variable to force color output
 ENV FORCE_COLOR=1
 ENV TERM=xterm-256color
 
 # Set working directory
 WORKDIR /riven
 
-# Copy frontend build from the previous stage
-COPY --from=frontend  /app/build /riven/frontend/build
-COPY --from=frontend  /app/node_modules /riven/frontend/node_modules
-COPY --from=frontend  /app/package.json /riven/frontend/package.json
-
 # Copy the virtual environment from the builder stage
 COPY --from=builder /app/.venv /app/.venv
 ENV VIRTUAL_ENV=/app/.venv
 ENV PATH="/app/.venv/bin:$PATH"
 
 # Copy the rest of the application code
-COPY backend/ /riven/backend
-COPY pyproject.toml poetry.lock /riven/backend/
+COPY src/ /riven/src
+COPY pyproject.toml poetry.lock /riven/src/
 COPY VERSION entrypoint.sh /riven/
 
 # Ensure entrypoint script is executable
 RUN chmod +x /riven/entrypoint.sh
 
-# Switch to fish shell
-SHELL ["fish", "--login"]
-
-ENTRYPOINT ["fish", "/riven/entrypoint.sh"]
+ENTRYPOINT ["/riven/entrypoint.sh"]
diff --git a/Dockerfile.slim b/Dockerfile.slim
@@ -1,4 +1,4 @@
-# Riven Backend Builder
+# Riven src Builder
 
 FROM python:3.11.9-alpine3.19 as Base
 LABEL name="Riven" \
@@ -36,16 +36,16 @@ COPY pyproject.toml poetry.lock* /riven/
 # Install Python dependencies
 RUN poetry install --without dev --no-root && rm -rf $POETRY_CACHE_DIR 
 
-# Copy backend code and other necessary files
-COPY backend/ /riven/backend
+# Copy src code and other necessary files
+COPY src/ /riven/src
 COPY VERSION entrypoint.sh /riven/
 
-RUN cd /riven/backend && poetry add nuitka && \
+RUN cd /riven/src && poetry add nuitka && \
     poetry run python3 -m nuitka --standalone --onefile --onefile-tempdir-spec=/onefile_%PID%_%TIME% --python-flag=nosite,-O --nofollow-import-to=pytest --clang --warn-implicit-exceptions --warn-unusual-code --prefer-source-code main.py
 
 FROM scratch
 
-COPY --from=Base /riven/backend/main.bin /main.bin
+COPY --from=Base /riven/src/main.bin /main.bin
 COPY VERSION /
 VOLUME /data
 COPY --from=Base /lib/ /lib/

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-0.7.6
+0.8.0
diff --git a/backend/program/cache.py b/backend/program/cache.py
diff --git a/backend/program/downloaders/__init__.py b/backend/program/downloaders/__init__.py
diff --git a/backend/program/downloaders/torbox.py b/backend/program/downloaders/torbox.py