Tagging system?

[Die4Ever]

Tags could maybe be like: open source, RPG, FPS

Maybe don't do tags for features like entrance randomizer, because features can change often and it would be impossible to maintain on this scale

Tags should be an enum in the yml schema validation so that we don't end up with typos or synonymous tags. We don't want to end up like Twitch's horrible tagging system.

[SmashManiac]

If tags are implemented. it would also allow the addition of filters. I believe such filters could even be implemented in pure CSS if needed.

Also, the tag category I received the most requests for in the past was for game platforms: NES, PC, etc.

[Die4Ever]

Tags for platforms might be a bit complicated with multi platform games where the game runs on multiple platforms but the randomizer only works for certain ones. Or even randomizers that work on more platforms than the original game did (like Duke Nukem 3D or Doom). And then there's the possibility of updates where newer versions of the randomizer might work on more platforms, updates from source ports/fan ports.

[Die4Ever]

how about this? would this be too much work for us to track?

diff --git a/src/schemaCheck.py b/src/schemaCheck.py
index d410c2a..11c5a2b 100644
--- a/src/schemaCheck.py
+++ b/src/schemaCheck.py
@@ -21,7 +21,48 @@ MaybeString = {
         {"type": "null"},
 ]}

-def randomizer_schema(modified: datetime):
+tags = {
+    "type": "array",
+    "minLength": 1,
+    "items": {
+        "type": "string",
+        "minLength": 1,
+        "enum": [
+            "RPG", "Platformer", "Shooter", "Strategy", "Puzzle",
+        ]
+    }
+}
+
+platforms = {
+    "type": "array",
+    "minLength": 1,
+    "items": {
+        "type": "string",
+        "minLength": 1,
+        "enum": [
+            "PC",
+            "NES", "SNES", "N64", "GameCube", "Wii", "Wii U", "Switch",
+            "GameBoy", "GBA", "DS", "3DS",
+            "Master System", "Genesis", "Saturn", "Dreamcast",
+            "PS1", "PS2", "PS3", "PS4", "PS5",
+            "PSP", "Vita",
+            "Xbox", "Xbox 360", "Xbox One", "Xbox Series",
+        ]
+    }
+}
+
+
+games_years = {
+    "type": "array",
+    "minLength": 1,
+    "items": {
+        "type": "number",
+        "minimum": 1900,
+        "maximum": datetime.now().year + 1,
+    }
+}
+
+def randomizer_schema(data, modified: datetime):
     ret = {
         "type": "object",
         "properties": {
@@ -43,18 +84,26 @@ def randomizer_schema(modified: datetime):
             "community": ValidString,
             "contact": ValidString,
             "added-date": {},
-            "info-updated": {}, # we update the info, but we don't keep track of when the randomizers receive patches/new versions
-            "opensource": {"type": "boolean"}
+            "info-updated": {}, # when we update our info, but we don't keep track of when the randomizers receive patches/new versions
+            "opensource": {"type": "boolean"},
+            "tags": tags,
+            "platforms": platforms,
+            "games-years": games_years,
         },
         "required": ["games", "identifier", "url" ],
         "additionalProperties": False,
     }
-    if modified > datetime(2024, 6, 16):
+    if modified >= datetime(2024, 6, 16):
         ret['required'].extend(('info-updated', 'added-date'))
     # other new requirements can be checked by the added-date and info-updated dates, and eventually we can drop the expensive call to get_modified_time
+    updated = data.get('info-updated')
+    if not updated:
+        return ret
+    if updated >= date(2024, 6, 18):
+        ret['required'].extend(('opensource', 'tags', 'platforms'))
     return ret

-def series_schema(modified: datetime):
+def series_schema(data, modified: datetime):
     ret = {
         "type": "object",
         "properties": {

diff --git a/src/series/Deus_Ex.yml b/src/series/Deus_Ex.yml
index 5ced01a..8848622 100644
--- a/src/series/Deus_Ex.yml
+++ b/src/series/Deus_Ex.yml
@@ -10,4 +10,14 @@ randomizers:
     - HX (mod)
     - Vanilla? Madder. (mod)
     identifier: Die4Ever's
-    url: https://github.com/Die4Ever/deus-ex-randomizer/
+    url: https://mods4ever.com/project/DXRando
+    tags:
+    - RPG
+    - Shooter
+    platforms:
+    - PC
+    games-years:
+    - 2000
+    opensource: true
+    added-date: 2024-05-25
+    info-updated: 2024-06-18

[SmashManiac]

The hard part is going to be to fill in the data for all of the entries the first time. I don't think it will be hard to track afterwards. Maybe they could be implemented in waves to work around that.

As to the structure itself:

• Most of the tags apply to individual games, not the associated randomizer. Restructuring it by game would also allow fetching the data from an external database, which would save a lot of time.
• The platform list does not cover all games on the list right now, such as Arcade (for Donkey Kong) and MSX (for Metal Gear). There may be more.
• I would rename the "tag" field to "genre", and add "Adventure" to it for Zelda games and other similar games.

[Die4Ever]

We already have the ability to do it in waves! That code diff shows that the new fields are only required according to the updated date of the item being checked. And we could walk that cutoff date back 1 day at a time to eventually fix all the old ones, and then delete the date check and make it always required.

Currently the data layout doesn't have a list of games, games are just a list of strings inside each randomizer. Would need some restructuring.

Tag would allow more flexibility than genre, alright I can't think of any examples right now.

[Die4Ever]

I'll add just opensource and commit it for now, just as an example

later we'll maybe just do an automated script to extract out lists of games

[Die4Ever]

here's some example restructuring

Deus_Ex.yml

name: Deus Ex
comment: null
sub-series: null
games:
    Deus Ex:
        genres: []
        platforms: []
        release_year: null
    GMDX (mod):
        genres: []
        platforms: []
        release_year: null
    HX (mod):
        genres: []
        platforms: []
        release_year: null
    Lay D Denton (mod):
        genres: []
        platforms: []
        release_year: null
    Revision (mod):
        genres: []
        platforms: []
        release_year: null
    Vanilla? Madder. (mod):
        genres: []
        platforms: []
        release_year: null
randomizers:
-   games:
    - Deus Ex
    - Lay D Denton (mod)
    - GMDX (mod)
    - Revision (mod)
    - HX (mod)
    - Vanilla? Madder. (mod)
    identifier: Die4Ever's
    url: https://mods4ever.com/project/DXRando
    opensource: true
    added-date: 2024-05-25
    info-updated: 2024-06-19

code diff

diff --git a/src/schemaCheck.py b/src/schemaCheck.py
index e4a1dd0..9bf4b17 100644
--- a/src/schemaCheck.py
+++ b/src/schemaCheck.py
@@ -7,6 +7,7 @@ import datetime as datetime_module
 from datetime import date
 from datetime import datetime
 import re
+import traceback

 # https://json-schema.org/learn/getting-started-step-by-step#going-deeper-with-properties
 # https://cswr.github.io/JsonSchema/spec/introduction/
@@ -22,6 +23,54 @@ MaybeString = {
         {"type": "null"},
 ]}

+genres = {
+    "type": "array",
+    "minItems": 0,
+    "items": {
+        "type": "string",
+        "enum": [
+            "Adventure", "RPG", "Platformer", "Shooter", "Strategy", "Puzzle",
+        ]
+    }
+}
+
+platforms = {
+    "type": "array",
+    "minItems": 0,
+    "items": {
+        "type": "string",
+        "enum": [
+            "PC",
+            "NES", "SNES", "N64", "GameCube", "Wii", "Wii U", "Switch",
+            "GameBoy", "GBA", "DS", "3DS",
+            "Master System", "Genesis", "Saturn", "Dreamcast",
+            "PS1", "PS2", "PS3", "PS4", "PS5",
+            "PSP", "Vita",
+            "Xbox", "Xbox 360", "Xbox One", "Xbox Series",
+            "Arcade", "MSX",
+        ]
+    }
+}
+
+games_schema = {
+    "type": "object",
+    "additionalProperties": {
+        "type": "object",
+        "properties": {
+            "genres": genres,
+            "platforms": platforms,
+            "release_year": {
+                "type": ["number", 'null'],
+                "minimum": 1900,
+                "maximum": datetime.now().year + 1
+            },
+            "sub-series": ValidString
+        },
+        "required": ["genres", "platforms", "release_year"],
+        "additionalProperties": False
+    }
+}
+
 def randomizer_schema(data):
     ret = {
         "type": "object",
@@ -56,6 +105,8 @@ def randomizer_schema(data):
         return ret
     if updated >= date(2024, 6, 19):
         ret['required'].append('opensource')
+    if updated >= date(2024, 7, 1):
+        ret['properties'].pop('sub-series') # deprecated, moved to games_schema        
     return ret

 def series_schema(data):
@@ -67,6 +118,7 @@ def series_schema(data):
             "sub-series": {
                 "type": ["array", "null"],
             },
+            "games": games_schema,
             "randomizers": {
                 "type": "array",
                 "minItems": 1,
@@ -117,6 +169,7 @@ def validateSeriesConfig(path: Path):
     failures = 0
     #modified = get_modified_time(path) # currently we don't need this expensive check 
     writeback = False
+    games = set()
     text = path.read_text()
     data = yaml.load(text, Loader=yaml.CLoader)

@@ -132,15 +185,21 @@ def validateSeriesConfig(path: Path):
     except ValidationError as e:
         failures += 1
         print('ERROR in', path, ': series definition -\n', e.path, e.message)
+        #raise # uncomment for bigger error messages

     for rando in data['randomizers']:
         try:
             validateRando(rando)
+            for game in rando.get('games', []):
+                if 'games' in data and game not in data['games']:
+                    raise ValidationError('game: ' + game + ' is not defined')
+                games.add(game)
         except ValidationError as e:
             failures += 1
             print('\nIn Randomizer definition:', rando)
             id = str(rando.get('game', ''))  + ' ' + str(rando.get('identifier', ''))  
             print('\nERROR in', path, ': randomizer definition', id, '-\n', e.path, e.message)
+            #raise # uncomment for bigger error messages

         # update rando data
         if not rando.get('info-updated'):
@@ -150,6 +209,15 @@ def validateSeriesConfig(path: Path):
             writeback = True
             rando['added-date'] = date(2024, 5, 25)

+    if 'games' not in data:
+        newdata: dict = data.copy()
+        newdata.pop('randomizers')
+        games = {key: {'genres':[], 'platforms':[], 'release_year': None} for key in sorted(games)}
+        newdata['games'] = games
+        newdata['randomizers'] = data['randomizers']
+        data = newdata
+        writeback=True
+
     if failures == 0 and writeback:
         out = yaml.dump(data, sort_keys=False, indent=4)
         path.write_text(out)
@@ -169,7 +237,9 @@ def validateYamlFiles():
             failures += new_failures
         except Exception as e:
             failures += 1
+            print(traceback.format_exc())
             print('\nERROR in', file, '-\n', e)
+            #raise # uncomment for bigger error messages

     print('\n\n')
     assert success > 0

[Die4Ever]

forgot to put the issue number in the commit messages...

d337f1d

7f923d5

[duncathan]

the obsolete field may warrant being rolled into the tagging system