Reproducing memory issue #4578

trinib · 2022-05-21T12:37:54Z

trinib
May 21, 2022

I posted a live preview of memory build up issue 4437#. It is running a python script for adding multiple blocklist URLs to AGH.

SCRIPT

import requests
import ssl
import json
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager

# +++ A SCRIPT TO ADD MULTIPLE BLOCKLIST AT ONCE IN ADGUARD +++ #

### RUN SCRIPT ###
# sudo python3 bulkurls.py

### SET IP / USERNAME / PASSWORD USING *YOUR* ADGUARD CREDENTIALS ###
host = "http://your_pi_ip:80" 
# Your user name
userName = "Yours"
# Your password
password = "Yours"

# My blocklist (add your own: https://github.com/T145/black-mirror) 
urls = [
"https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt",
"https://adaway.org/hosts.txt",
"https://someonewhocares.org/hosts/zero/hosts",
"https://raw.githubusercontent.com/DandelionSprout/adfilt/master/GameConsoleAdblockList.txt",
"https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hosts",
"https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV-AGH.txt",
"https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt",
"https://raw.githubusercontent.com/Spam404/lists/master/main-blacklist.txt",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=adblockplus&showintro=1&mimetype=plaintext",
"https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt",
"https://raw.githubusercontent.com/durablenapkin/scamblocklist/master/adguard.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-agh-online.txt",
"https://paulgb.github.io/BarbBlock/blacklists/hosts-file.txt",
"https://abp.oisd.nl/",
"https://v.firebog.net/hosts/Easyprivacy.txt",
"https://v.firebog.net/hosts/Prigent-Ads.txt",
"https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt",
"https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.2o7Net/hosts",
"https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt",
"https://v.firebog.net/hosts/static/w3kbl.txt",
"https://v.firebog.net/hosts/AdguardDNS.txt",
"https://v.firebog.net/hosts/Admiral.txt",
"https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt",
"https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt",
"https://v.firebog.net/hosts/Easylist.txt",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext",
"https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts",
"https://raw.githubusercontent.com/Kees1958/W3C_annual_most_used_survey_blocklist/master/TOP_EU_US_Ads_Trackers_HOST",
"https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt",
"https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt",
"https://v.firebog.net/hosts/Prigent-Crypto.txt",
"https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt",
"https://phishing.army/download/phishing_army_blocklist_extended.txt",
"https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt",
"https://v.firebog.net/hosts/Shalla-mal.txt",
"https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts",
"https://urlhaus.abuse.ch/downloads/hostfile/",
"https://raw.githubusercontent.com/neodevpro/neodevhost/master/lite_host",
"https://raw.githubusercontent.com/neodevpro/neodevhost/master/lite_adblocker",
"https://perflyst.github.io/PiHoleBlocklist/AmazonFireTV.txt",
"https://perflyst.github.io/PiHoleBlocklist/android-tracking.txt",
"https://perflyst.github.io/PiHoleBlocklist/regex.list",
"https://perflyst.github.io/PiHoleBlocklist/SessionReplay.txt",
"https://blocklistproject.github.io/Lists/phishing.txt",
"https://blocklistproject.github.io/Lists/tracking.txt",
"https://raw.githubusercontent.com/ftpmorph/ftprivacy/master/blocklists/smartphone-ads-tracking.txt",
"https://raw.githubusercontent.com/CleanMachine1/AdlistTXTS/main/adservers.txt",
"https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt",
"https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_15_DnsFilter/filter.txt",
"https://easylist-downloads.adblockplus.org/easylist.txt",
"https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_11_Mobile/filter.txt",
"https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_3_Spyware/filter.txt",
"https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_2_English/filter.txt",
"https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_4_Social/filter.txt",
"https://easylist.to/easylist/easyprivacy.txt",
"https://easylist.to/easylist/fanboy-annoyance.txt",
"https://easylist.to/easylist/fanboy-social.txt",
"https://anti-ad.net/easylist.txt",
"https://hosts.anudeep.me/mirror/adservers.txt",
"https://raw.githubusercontent.com/ookangzheng/dbl-oisd-nl/master/abp_light.txt",
"https://raw.githubusercontent.com/ookangzheng/dbl-oisd-nl/master/dbl_light.txt",
"https://raw.githubusercontent.com/ookangzheng/dbl-oisd-nl/master/hosts_light.txt",
"https://raw.githubusercontent.com/ookangzheng/dbl-oisd-nl/master/dnsmasq_light.txt",
"https://raw.githubusercontent.com/ookangzheng/dbl-oisd-nl/master/rpz_basic.rpz",
"https://raw.githubusercontent.com/smed79/blacklist/master/hosts.txt",
"https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt",
"https://raw.githubusercontent.com/azet12/KADhosts/master/KADhosts.txt",
"https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-domains-ACTIVE.txt",
"https://gitlab.com/quidsup/notrack-blocklists/-/raw/master/malware.list",
"https://gitlab.com/quidsup/notrack-blocklists/-/raw/master/trackers.list",
"https://raw.githubusercontent.com/yous/YousList/master/hosts.txt",
"https://zerodot1.gitlab.io/CoinBlockerLists/list.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/-/raw/master/list_browser.txt",
"https://raw.githubusercontent.com/mitchellkrogza/Badd-Boyz-Hosts/master/domains",
"https://easylist-downloads.adblockplus.org/easyprivacy.txt",
"https://raw.githubusercontent.com/privacy-protection-tools/anti-AD/master/anti-ad-easylist.txt",
"https://www.i-dont-care-about-cookies.eu/abp/",
"https://hblock.molinero.dev/hosts",
"http://winhelp2002.mvps.org/hosts.txt",
"https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts",
"https://raw.githubusercontent.com/VeleSila/yhosts/master/hosts.txt",
"https://iplists.firehol.org/files/alienvault_reputation.ipset",
"https://orca.pet/notonmyshift/domains.txt",
"https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/firewall/spy.txt",
"https://raw.githubusercontent.com/d43m0nhLInt3r/socialblocklists/master/Windows/windowstelemetryblocklist.txt",
"https://raw.githubusercontent.com/d43m0nhLInt3r/socialblocklists/master/MobileAppAds/appadsblocklist.txt",
"https://raw.githubusercontent.com/d43m0nhLInt3r/socialblocklists/master/Android/androidblocklist.txt",
"https://raw.githubusercontent.com/twcau/AdblockRules/master/MurdochList",
"https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews/hosts",
"https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling/hosts",
"https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Dead/hosts",
"https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts",
"https://raw.githubusercontent.com/mitchellkrogza/Badd-Boyz-Hosts/master/hosts",
"https://winhelp2002.mvps.org/hosts.txt",
"https://raw.githubusercontent.com/notracking/hosts-blocklists/master/adblock/adblock.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/mini/unbound.conf",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/mini/hosts.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/mini/adblock.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/mini/hosts.win",
"https://blocklistproject.github.io/Lists/adguard/ransomware-ags.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/filters.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/privacy.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/unbreak.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter.txt",
"https://abp.oisd.nl/basic/",
"https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareAdGuardHome.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-ag.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resource-abuse.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/badware.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easyprivacy/easyprivacy_general.txt",
"https://secure.fanboy.co.nz/fanboy-cookiemonster.txt",
"https://easylist-downloads.adblockplus.org/easyprivacy+easylist.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/annoyances.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/filters-2022.txt",
"https://secure.fanboy.co.nz/fanboy-annoyance.txt",
"https://easylist.to/easylist/easylist.txt",
"https://curbengh.github.io/malware-filter/urlhaus-filter.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-vivaldi.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-domains.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-agh.txt",
"https://curben.gitlab.io/malware-filter/urlhaus-filter-hosts.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easylist/easylist_adservers.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easylist/easylist_general_block.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easylist/easylist_adservers_popup.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easyprivacy/easyprivacy_trackingservers.txt",
"https://raw.githubusercontent.com/angad/webPriority/master/easylist/easylist/easylist_general_block.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/thirdparties/easylist-downloads.adblockplus.org/easylist.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/thirdparties/easylist-downloads.adblockplus.org/easyprivacy.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/filters-2021.txt",
"https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/filters-2020.txt",
"https://raw.githubusercontent.com/easylist/easylist/master/easyprivacy/easyprivacy_thirdparty.txt",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=adblock&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=littlesnitch&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=junkbuster&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=unbound&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=iptables&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=msfilter&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=operafilter&showintro=0&mimetype=plaintext",
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=machosts&showintro=0&mimetype=plaintext",
"https://raw.githubusercontent.com/BlackJack8/iOSAdblockList/master/Regular%20Hosts.txt",
"https://raw.githubusercontent.com/uniartisan/adblock_list/master/adblock.txt",
"https://raw.githubusercontent.com/uniartisan/adblock_list/master/adblock_lite.txt",
"https://raw.githubusercontent.com/uniartisan/adblock_list/master/adblock_plus.txt",
"https://raw.githubusercontent.com/uniartisan/adblock_list/master/adblock_privacy.txt",
"https://raw.githubusercontent.com/TonyRL/blocklist/master/hosts",
"https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/AmazonFireTV.txt",
"https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt",
"https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt",
"https://raw.githubusercontent.com/ShadowWhisperer/BlockLists/master/Lists/Tracking",
"https://raw.githubusercontent.com/ShadowWhisperer/BlockLists/master/Lists/Malware",
"https://raw.githubusercontent.com/hemiipatu/PiHoleBlocklists/master/blocklists/advertisement.txt",
"https://raw.githubusercontent.com/hemiipatu/PiHoleBlocklists/master/blocklists/phishing.txt",
"https://raw.githubusercontent.com/DandelionSprout/adfilt/master/AdGuard%20Home%20Compilation%20List/AdGuardHomeCompilationList.txt",
"https://raw.githubusercontent.com/ftpmorph/ftpihole/master/blocklists/smartphone-ads-tracking.txt",
"https://raw.githubusercontent.com/mhxion/pornaway/master/hosts/porn_ads.txt",
"https://raw.githubusercontent.com/privacy-protection-tools/anti-AD/master/anti-ad-domains.txt",
"https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt",
"http://security-research.dyndns.org/pub/malware-feeds/ponmocup-infected-domains-CIF-latest.txt",
"https://anti-ad.net/domains.txt",
"https://block.energized.pro/assets/sources/filter/better-fyi-trackers.txt",
"https://block.energized.pro/assets/sources/filter/sinfonietta-porn.txt",
"https://block.energized.pro/extensions/xtreme/formats/domains.txt",
"https://blocklistproject.github.io/Lists/alt-version/everything-nl.txt",
"https://blocklistproject.github.io/Lists/alt-version/smart-tv-nl.txt",
"https://blocklistproject.github.io/Lists/alt-version/vaping-nl.txt",
"https://blocklistproject.github.io/Lists/alt-version/whatsapp-nl.txt",
"https://blokada.org/blocklists/ddgtrackerradar/standard/hosts.txt",
"https://cinsscore.com/list/ci-badguys.txt",
"https://data.netlab.360.com/feeds/dga/dga.txt",
"https://dataplane.org/sipinvitation.txt",
"https://dataplane.org/sipquery.txt",
"https://dataplane.org/sipregistration.txt",
"https://dataplane.org/sshpwauth.txt",
"https://dataplane.org/vncrfb.txt",
"https://fanboy.co.nz/fanboy-antifonts.txt",
"https://fanboy.co.nz/fanboy-cookiemonster.txt",
"https://fanboy.co.nz/fanboy-indian.txt",
"https://fanboy.co.nz/r/fanboy-ultimate.txt",
"https://feeds.dshield.org/block.txt",
"https://filter.futa.gg/hosts.txt",
"https://filtri-dns.ga/filtri.txt",
"https://getblackbird.net/blacklist/hosts.txt",
"https://gitlab.com/The_Quantum_Alpha/the-quantum-ad-list/-/raw/master/Individual%20lists/Extreme%20lists/The_Quantum_Extreme_Youtube_Ad-List.txt",
"https://gitlab.com/The_Quantum_Alpha/the-quantum-ad-list/-/raw/master/Individual%20lists/The_Quantum_Abuse-List.txt",
"https://gitlab.com/The_Quantum_Alpha/the-quantum-ad-list/-/raw/master/Individual%20lists/The_Quantum_Privacy-list.txt",
"https://gitlab.com/The_Quantum_Alpha/the-quantum-ad-list/-/raw/master/Individual%20lists/The_Quantum_Simply-ads-list.txt",
"https://gitlab.com/The_Quantum_Alpha/the-quantum-ad-list/-/raw/master/Individual%20lists/The_Quantum_Youtube-Ads-List.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/-/raw/master/list.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/-/raw/master/list_optional.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/-/raw/master/white_list.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/raw/master/list.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/raw/master/list_browser.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/raw/master/list_optional.txt",
"https://gitlab.com/ZeroDot1/CoinBlockerLists/raw/master/white_list.txt",
"https://gitlab.com/quidsup/ntrk-tracker-radar/-/raw/master/ddg_tracker_radar_confirmed.txt",
"https://hostfiles.frogeye.fr/firstparty-trackers.txt",
"https://hostfiles.frogeye.fr/multiparty-trackers.txt",
"https://hosts.anudeep.me/mirror/CoinMiner.txt",
"https://hosts.anudeep.me/mirror/facebook.txt",
"https://hosts.anudeep.me/mirror/miscellaneous/false-positives.txt",
"https://kriskintel.com/feeds/ktip_covid_domains.txt",
"https://kriskintel.com/feeds/ktip_malicious_Ips.txt",
"https://kriskintel.com/feeds/ktip_malicious_domains.txt",
"https://kriskintel.com/feeds/ktip_ransomware_feeds.txt",
"https://lists.blocklist.de/lists/all.txt",
"https://malsilo.gitlab.io/feeds/dumps/domain_list.txt",
"https://mirai.security.gives/data/ip_list.txt",
"https://myip.ms/files/bots/live_webcrawlers.txt",
"https://openphish.com/feed.txt",
"https://orca.pet/notonmyshift/hosts.txt",
"https://osint.digitalside.it/Threat-Intel/lists/latestips.txt",
"https://public-dns.info/nameservers.txt",
"https://raw.githubusercontent.com/4skinSkywalker/Anti-Porn-HOSTS-File/master/HOSTS.txt",
"https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/combined_disguised_trackers_justdomains.txt",
"https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/activation.txt",
"https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/adservers-and-trackers.txt",
"https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/fake-domains.txt",
"https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/search-blacklist.txt",
"https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/unwanted-iranian.txt",
"https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareDomains.txt",
"https://raw.githubusercontent.com/DandelionSprout/adfilt/master/NorwegianList.txt",
"https://raw.githubusercontent.com/EnergizedProtection/unblock/master/basic/formats/domains.txt",
"https://raw.githubusercontent.com/Ewpratten/youtube_ad_blocklist/master/blocklist.txt",
"https://raw.githubusercontent.com/FiltersHeroes/KADhosts/master/KADomains.txt",
"https://raw.githubusercontent.com/FutaGuard/FutaFilter/master/hosts.txt",
"https://raw.githubusercontent.com/List-KR/List-KR/master/filter.txt",
"https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/hole_cert_pl.txt",
"https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/hostfile.txt",
"https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SessionReplay.txt",
"https://raw.githubusercontent.com/TheAntiSocialEngineer/AntiSocial-BlockList-UK-Community/main/UK-Community.txt",
"https://raw.githubusercontent.com/allendema/noplaylist/main/NoPlayList.txt",
"https://raw.githubusercontent.com/anudeepND/blacklist/master/CoinMiner.txt",
"https://raw.githubusercontent.com/anudeepND/blacklist/master/facebook.txt",
"https://raw.githubusercontent.com/anudeepND/blacklist/master/miscellaneous/false-positives.txt",
"https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/submit_here/exclude_for_all.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/submit_here/include_for_Pro_Xtra_only.txt",
"https://raw.githubusercontent.com/badmojr/1Hosts/master/submit_here/include_for_all.txt",
"https://raw.githubusercontent.com/blocklistproject/Lists/master/alt-version/everything-nl.txt",
"https://raw.githubusercontent.com/blocklistproject/Lists/master/alt-version/smart-tv-nl.txt",
"https://raw.githubusercontent.com/blocklistproject/Lists/master/alt-version/vaping-nl.txt",
"https://raw.githubusercontent.com/blocklistproject/Lists/master/alt-version/whatsapp-nl.txt",
"https://raw.githubusercontent.com/blokadaorg/landing-github-pages/master/blocklists/ddgtrackerradar/standard/hosts.txt",
"https://raw.githubusercontent.com/bongochong/CombinedPrivacyBlockLists/master/NoFormatting/BlacklistedDomains.txt",
"https://raw.githubusercontent.com/craiu/mobiletrackers/master/list.txt",
"https://raw.githubusercontent.com/craiu/mobiletrackers/master/windowslist.txt",
"https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/extra_v6.txt",
"https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy_v6.txt",
"https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/update_v6.txt",
"https://raw.githubusercontent.com/davidonzo/Threat-Intel/master/lists/latestdomains.txt",
"https://raw.githubusercontent.com/davidonzo/Threat-Intel/master/lists/latestips.txt",
"https://raw.githubusercontent.com/durablenapkin/block/master/luminati.txt",
"https://raw.githubusercontent.com/durablenapkin/block/master/streaming.txt",
"https://raw.githubusercontent.com/durablenapkin/scamblocklist/master/hosts.txt",
"https://raw.githubusercontent.com/finnish-easylist-addition/finnish-easylist-addition/master/Finland_adb.txt",
"https://raw.githubusercontent.com/gioxx/xfiles/master/filtri.txt",
"https://raw.githubusercontent.com/gioxx/xfiles/master/siteblock.txt",
"https://raw.githubusercontent.com/heradhis/indonesianadblockrules/master/subscriptions/abpindo.txt",
"https://raw.githubusercontent.com/infinitytec/blocklists/master/ads-and-trackers.txt",
"https://raw.githubusercontent.com/infinitytec/blocklists/master/medicalpseudoscience.txt",
"https://raw.githubusercontent.com/infinitytec/blocklists/master/scams-and-phishing.txt",
"https://raw.githubusercontent.com/jakejarvis/ios-trackers/master/blocklist.txt",
"https://raw.githubusercontent.com/lassekongo83/Frellwits-filter-lists/master/Frellwits-Swedish-Hosts-File.txt",
"https://raw.githubusercontent.com/lightswitch05/hosts/master/docs/lists/ads-and-tracking-extended.txt",
"https://raw.githubusercontent.com/lightswitch05/hosts/master/docs/lists/amp-hosts-extended.txt",
"https://raw.githubusercontent.com/lightswitch05/hosts/master/docs/lists/facebook-extended.txt",
"https://raw.githubusercontent.com/lightswitch05/hosts/master/docs/lists/hate-and-junk-extended.txt",
"https://raw.githubusercontent.com/lightswitch05/hosts/master/docs/lists/tracking-aggressive-extended.txt",
"https://raw.githubusercontent.com/marcusminus/Orthrus-BlockList/master/domains.txt",
"https://raw.githubusercontent.com/matomo-org/referrer-spam-list/master/spammers.txt",
"https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-IPs-ACTIVE.txt",
"https://raw.githubusercontent.com/notracking/hosts-blocklists/master/dnscrypt-proxy/dnscrypt-proxy.blacklist.txt",
"https://raw.githubusercontent.com/rimu/no-qanon/master/etc_hosts.txt",
"https://raw.githubusercontent.com/scafroglia93/blocklists/master/blocklists-certego.txt",
"https://raw.githubusercontent.com/stamparm/aux/master/maltrail-malware-domains.txt",
"https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt",
"https://rescure.me/covid.txt",
"https://rescure.me/malware/clop.txt",
"https://rescure.me/malware/ekans.txt",
"https://rescure.me/malware/maze.txt",
"https://rescure.me/malware/netwalker.txt",
"https://rescure.me/malware/ryuk.txt",
"https://rescure.me/malware/sodinokibi.txt",
"https://rescure.me/malware/try2cry.txt",
"https://rescure.me/malware/wastedlocker.txt",
"https://rescure.me/rescure_blacklist.txt",
"https://rescure.me/rescure_domain_blacklist.txt",
"https://schakal.ru/hosts/alive_hosts.txt",
"https://secure.fanboy.co.nz/fanboy-antifonts.txt",
"https://secure.fanboy.co.nz/fanboy-indian.txt",
"https://secure.fanboy.co.nz/r/fanboy-ultimate.txt",
"https://tgc.cloud/downloads/hosts.alive.txt",
"https://threatcrowd.org/feeds/domains.txt",
"https://threatcrowd.org/feeds/ips.txt",
"https://threatcrowd.org/inc/blacklist.txt",
"https://www.binarydefense.com/banlist.txt",
"https://www.blocklist.de/downloads/export-ips_all.txt",
"https://www.botvrij.eu/data/misp.text_ip-dst.ADMIN.txt",
"https://www.github.developerdan.com/hosts/lists/amp-hosts-extended.txt",
"https://www.github.developerdan.com/hosts/lists/facebook-extended.txt",
"https://www.github.developerdan.com/hosts/lists/hate-and-junk-extended.txt",
"https://www.github.developerdan.com/hosts/lists/tracking-aggressive-extended.txt",
"https://www.joewein.net/dl/bl/dom-bl.txt",
"https://www.spamhaus.org/drop/drop.txt",
"https://www.spamhaus.org/drop/edrop.txt",
"https://www.sunshine.it/blacklist.txt",
"https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt",
"https://www.threatsourcing.com/dnall-free.txt",
"https://www.threatsourcing.com/ipall-free.txt",
"https://zerodot1.gitlab.io/CoinBlockerLists/list_browser.txt",
"https://zerodot1.gitlab.io/CoinBlockerLists/list_optional.txt",
"https://zerodot1.gitlab.io/CoinBlockerLists/white_list.txt",
"https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt"
]

############ End Of Edits #################

# Open TLSv1 Adapter
class MyAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       ssl_version=ssl.PROTOCOL_TLSv1)

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0'}     

s = requests.Session()
s.mount(host, MyAdapter())
x = s.post(host + "/control/login", json.dumps({"name": userName, "password" : password}), headers=headers )
print(x.text)

for u in urls:
	filterObj = json.dumps({'url':u, "name":u,"whitelist":False})
	print(filterObj)
	x = s.post(host + "/control/filtering/add_url", data = filterObj, headers=headers)
	print(x.text)

I hope this can be a stress test and useful for anyone willing to get to the bottom of this memory problem at a faster pace 🙏

trinib · 2022-05-22T11:13:38Z

trinib
May 22, 2022
Author

I tried it on A VPS and the python script runs without freezing but AGH memory do not drop even after reboot

after reboot

EDITED

I added blocklist with fewer URLs(still alot) and after reboot it remains around 35% ram and AGH starts up and it's stable. So I guess it is AGH blocklist url part cause memory issue🤷‍♂️

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reproducing memory issue #4578

{{title}}

Replies: 1 comment

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Reproducing memory issue #4578

trinib May 21, 2022

Replies: 1 comment

trinib May 22, 2022 Author

EDITED

trinib
May 21, 2022

trinib
May 22, 2022
Author