Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release to main #42

Merged
merged 18 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# shellcheck disable=SC2148

use nix
49 changes: 29 additions & 20 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,29 +19,29 @@ jobs:
echo "TAG=${{ github.head_ref || github.ref_name }}" >> $GITHUB_ENV

- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Install Nix
uses: cachix/install-nix-action@v22
uses: cachix/install-nix-action@v24
with:
nix_path: nixpkgs=channel:nixos-23.05
nix_path: nixpkgs=channel:nixpkgs-23.11-darwin
extra_nix_config: |
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
substituters = https://cache.nixos.org/

- name: Install dependencies
run: |
nix-shell --run "true"
nix-shell --pure --run true

- name: Compile cython
- name: Build Cython
run: |
nix-shell --run "pipenv run make"
nix-shell --pure --run cython-build

- name: Build
- name: Build Docker image
run: |
echo "IMAGE_PATH=$(nix-build --no-out-link)" >> $GITHUB_ENV

- name: Set up SSH
- name: Configure SSH
run: |
mkdir -p ~/.ssh
echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa && chmod 600 ~/.ssh/id_rsa
Expand All @@ -53,25 +53,34 @@ jobs:
IdentityFile ~/.ssh/id_rsa
" > ~/.ssh/config

- name: Upload image
- name: Upload Docker image
run: |
scp $IMAGE_PATH remote:~
scp "${{ env.IMAGE_PATH }}" remote:~

- name: Deploy
run: |
ssh remote <<EOF
ssh remote <<\EOF
set -e
tag="${{ env.TAG }}"
image_filename="$(basename "${{ env.IMAGE_PATH }}")"

echo "Loading docker image"
docker load < $(basename $IMAGE_PATH) && rm $(basename $IMAGE_PATH)
cleanup() {
cd ~
echo "Cleaning up"
rm -f "$image_filename"
}

echo "Fetching latest changes from the repository"
cd $TAG
git fetch origin $TAG
git checkout $TAG
git reset --hard origin/$TAG
trap cleanup EXIT

echo "Loading Docker image"
docker load < "$image_filename"

echo "Fetching latest changes from the git repository"
cd "$tag"
git fetch origin "$tag"
git checkout "$tag"
git reset --hard "origin/$tag"

echo "Restarting containers"
export TAG=$TAG
docker compose --env-file envs/compose/$TAG.env up -d
TAG="$tag" docker compose --env-file "envs/compose/$tag.env" up -d
EOF
4 changes: 3 additions & 1 deletion Caddyfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
:443 {
reverse_proxy cache:80
reverse_proxy cache:80 {
header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
}

tls /cert/ssl.crt /cert/ssl.key {
client_auth {
Expand Down
20 changes: 0 additions & 20 deletions Makefile

This file was deleted.

44 changes: 0 additions & 44 deletions Pipfile

This file was deleted.

1,397 changes: 0 additions & 1,397 deletions Pipfile.lock

This file was deleted.

10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ It's your shortcut to seamless dependency management and reproducible environmen
It will save you lots of time and spare you from unnecessary stress.

```sh
# Install dependencies and packages
# Install dependencies and enter shell
nix-shell

# Start up the database
make dev-start
dev-start

# Launch the web server
uvicorn main:app
Expand All @@ -47,8 +47,8 @@ Navigate to http://localhost:8000 to access the web app locally.

```sh
# Terminate the database
make dev-stop
dev-stop

# Purge data
rm -r data
# Purge database
dev-clean
```
17 changes: 7 additions & 10 deletions api/v1/node.py
Original file line number Diff line number Diff line change
@@ -1,33 +1,30 @@
import re
from datetime import datetime

from dateutil import tz
from fastapi import APIRouter, HTTPException
from timezonefinder import TimezoneFinder
from pytz import timezone
from tzfpy import get_tz

from models.lonlat import LonLat
from states.aed_state import AEDStateDep
from states.photo_state import PhotoStateDep

router = APIRouter()

tf = TimezoneFinder()

photo_id_re = re.compile(r'view/(?P<id>\S+)\.')


def _get_timezone(lonlat: LonLat) -> tuple[str | None, str | None]:
timezone_name = tf.timezone_at(lng=lonlat.lon, lat=lonlat.lat)
timezone_name: str | None = get_tz(lonlat.lon, lonlat.lat)
timezone_offset = None

if timezone_name:
try:
dt = datetime.now(tz=tz.gettz(timezone_name))
dt = datetime.now(tz=timezone(timezone_name))
offset = dt.strftime('%z')
timezone_offset = f'UTC{offset[:3]}:{offset[3:]}'
except Exception:
timezone_offset = None
else:
timezone_offset = None
except Exception: # noqa: S110
pass

return timezone_name, timezone_offset

Expand Down
7 changes: 3 additions & 4 deletions api/v1/photos.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
from states.aed_state import AEDStateDep
from states.photo_report_state import PhotoReportStateDep
from states.photo_state import PhotoStateDep
from utils import upgrade_https

router = APIRouter(prefix='/photos')

Expand Down Expand Up @@ -77,7 +76,7 @@ async def upload(
raise HTTPException(403, 'User has an active block on OpenStreetMap')

photo_info = await photo_state.set_photo(node_id, str(osm_user['id']), file)
photo_url = upgrade_https(str(request.url_for('view', id=photo_info.id)))
photo_url = f'{request.base_url}api/v1/photos/view/{photo_info.id}.webp'

node_xml = await osm.get_node_xml(node_id)

Expand Down Expand Up @@ -110,7 +109,7 @@ async def report_rss(
fg = FeedGenerator()
fg.title('AED Photo Reports')
fg.description('This feed contains a list of recent AED photo reports')
fg.link(href=upgrade_https(str(request.url)), rel='self')
fg.link(href=str(request.url), rel='self')

for report in await photo_report_state.get_recent_reports():
info = await photo_state.get_photo_by_id(report.photo_id)
Expand All @@ -130,7 +129,7 @@ async def report_rss(
),
type='CDATA',
)
fe.link(href=upgrade_https(f'{request.base_url}api/v1/photos/view/{report.photo_id}.webp'))
fe.link(href=f'{request.base_url}api/v1/photos/view/{report.photo_id}.webp')
fe.published(datetime.utcfromtimestamp(report.timestamp).astimezone(tz=UTC))

return Response(content=fg.rss_str(pretty=True), media_type='application/rss+xml')
2 changes: 1 addition & 1 deletion config.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from pyproj import Transformer

NAME = 'openaedmap-backend'
VERSION = '2.3'
VERSION = '2.4'
VERSION_TIMESTAMP = 0
CREATED_BY = f'{NAME} {VERSION}'
WEBSITE = 'https://openaedmap.org'
Expand Down
23 changes: 9 additions & 14 deletions default.nix
Original file line number Diff line number Diff line change
@@ -1,28 +1,23 @@
{ pkgsnix ? import ./pkgs.nix
, pkgs ? pkgsnix.pkgs
, unstable ? pkgsnix.unstable
}:
{ pkgs ? import <nixpkgs> { }, ... }:

with pkgs; let
let
envTag = builtins.getEnv "TAG";

shell = import ./shell.nix {
inherit pkgs;
inherit unstable;
isDocker = true;
isDevelopment = false;
};

python-venv = buildEnv {
python-venv = pkgs.buildEnv {
name = "python-venv";
paths = [
(runCommand "python-venv" { } ''
(pkgs.runCommand "python-venv" { } ''
mkdir -p $out/lib
cp -r "${./.venv/lib/python3.11/site-packages}"/* $out/lib
cp -r "${./.venv/lib/python3.12/site-packages}"/* $out/lib
'')
];
};
in
dockerTools.buildLayeredImage {
with pkgs; dockerTools.buildLayeredImage {
name = "backend";
tag = if envTag != "" then envTag else "latest";

Expand All @@ -33,20 +28,20 @@ dockerTools.buildLayeredImage {
mkdir tmp
mkdir app && cd app
cp "${./.}"/LICENSE .
cp "${./.}"/Makefile .
cp "${./.}"/*.py .
cp -r "${./.}"/api .
cp -r "${./.}"/cython_lib .
cp -r "${./.}"/middlewares .
cp -r "${./.}"/models .
cp -r "${./.}"/states .
export PATH="${lib.makeBinPath shell.buildInputs}":$PATH
${shell.shellHook}
'';

config = {
WorkingDir = "/app";
Env = [
"LD_LIBRARY_PATH=${lib.makeLibraryPath shell.buildInputs}"
"LD_LIBRARY_PATH=${lib.makeLibraryPath shell.libraries}"
"PYTHONPATH=${python-venv}/lib"
"PYTHONUNBUFFERED=1"
"PYTHONDONTWRITEBYTECODE=1"
Expand Down
17 changes: 3 additions & 14 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,20 +75,9 @@ services:
depends_on:
- app

volumes:
- ./default.vcl:/etc/varnish/default.vcl:ro
- ./data/cache:/var/lib/varnish

https:
image: caddy:alpine
restart: unless-stopped

depends_on:
- cache

ports:
- ${LISTEN:-443}:443
- ${LISTEN:-80}:80

volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- ./cert:/cert:ro
- ./default.vcl:/etc/varnish/default.vcl:ro
- ./data/cache:/var/lib/varnish
2 changes: 1 addition & 1 deletion envs/compose/dev.env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
LISTEN=49223
LISTEN=127.0.0.1:49223
WORKERS=4
2 changes: 1 addition & 1 deletion envs/compose/main.env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
LISTEN=49222
LISTEN=127.0.0.1:49222
WORKERS=12
5 changes: 0 additions & 5 deletions pkgs.nix

This file was deleted.

Loading