Speedup test_urls

A bunch of tweaks to make the URLs checker a bit faster.

• Before: 27m57s https://gitlab.apertis.org/docs/apertis-website/-/jobs/1006532
• After: 6m44s https://gitlab.apertis.org/docs/apertis-website/-/jobs/1006736

The URL check now also run in parallel with everything else, so the time for the whole pipeline went from 34m55s to 6m45s.

The URLs below still fail for unclear reasons:

• http://nl.farnell.com/ept/402-51501-51/conn-stacking-rcpt-440pos-0-5mm/dp/2474236
• http://www.amazon.co.uk/gp/product/B0033CA2XM
• http://www.amazon.co.uk/gp/product/B002VGQQAE
• http://nl.farnell.com/samtec/qte-040-03-l-d-a/header-dual-80way/dp/1667888
• https://qa.apertis.org/flatdeb-demo.html
• https://www.amazon.com/TRENDnet-Converter-Installation-Universal-TU-S9/dp/B0007T27H8
• https://www.renesas.com/us/en/products/automotive-products/automotive-system-chips-socs/r-car-h3-m3-starter-kit

The only actually broken URL is:

• http://photon.komoot.de/

added 1 commit

• 93700c29 - test_urls: Print exceptions when validating URLs

Compare with previous version

added 1 commit

• d904b5f3 - test_urls: Print exceptions when validating URLs

Compare with previous version

added 1 commit

• 3aed96ad - test_urls: Print exceptions when validating URLs

Compare with previous version

added 1 commit

• 44d9efa8 - test_urls: Always print the HTTP status code fore each URL

Compare with previous version

added 2 commits

• 7a6076b9 - test_urls: Print exceptions when validating URLs
• 24a1a044 - test_urls: Always print the HTTP status code fore each URL

Compare with previous version

added 5 commits

• 3729e93d - test_urls: Use a threadpool for the HTTP requests
• 6d37fb3a - test_urls: Summarize the broken URLs at the end
• 92cf18d9 - test_urls: Use a set to avoid O(N) insertion
• 4d51b96d - test_urls: Print exceptions when validating URLs
• 368e0b9a - test_urls: Always print the HTTP status code fore each URL

Compare with previous version

added 1 commit

• 919a68dc - test_urls: Use a session to retry requests

Compare with previous version

added 1 commit

• a7a32b6f - test_urls: Run under Bullseye, improve reliability

Compare with previous version

added 7 commits

• 9763856e - test_urls: Use a threadpool for the HTTP requests
• 6b6930f6 - test_urls: Summarize the broken URLs at the end
• d9a1b68c - test_urls: Use a set to avoid O(N) insertion
• 0066fa0a - test_urls: Print exceptions when validating URLs
• c818a1bb - test_urls: Always print the HTTP status code fore each URL
• 43a47378 - test_urls: Use a session to retry requests
• 6d87c269 - test_urls: Run under Bullseye, improve reliability

Compare with previous version

added 7 commits

• e11371ee - test_urls: Use a threadpool for the HTTP requests
• 5dcef862 - test_urls: Summarize the broken URLs at the end
• 5c4ddef5 - test_urls: Use a set to avoid O(N) insertion
• 51762c9f - test_urls: Print exceptions when validating URLs
• 18d4ebc8 - test_urls: Always print the HTTP status code fore each URL
• 532f46a9 - test_urls: Use a session to retry requests
• bf24376c - test_urls: Run under Bullseye, improve reliability

Compare with previous version

added 1 commit

• 364c257c - test_urls: Debug "Invalid Retry-After" error

Compare with previous version

marked this merge request as draft

changed the description

added 1 commit

• 6322bcb1 - test_urls: Debug "Invalid Retry-After" error

Compare with previous version

added 8 commits

• 434896f9 - test_urls: Use a threadpool for the HTTP requests
• 9925848c - test_urls: Summarize the broken URLs at the end
• 10ad628e - test_urls: Use a set to avoid O(N) insertion
• be54df3c - test_urls: Print exceptions when validating URLs
• 225d405f - test_urls: Always print the HTTP status code fore each URL
• eb58ecd5 - test_urls: Use a session to retry requests
• c130174a - test_urls: Run under Bullseye, improve reliability
• 5e67ece2 - test_urls: Debug "Invalid Retry-After" error

Compare with previous version

added 6 commits

• 3f1c1a5b - test_urls: Print exceptions when validating URLs
• b66cc6d9 - test_urls: Always print the HTTP status code fore each URL
• 220b220d - test_urls: Use a session to retry requests
• 27e093df - test_urls: Run under Bullseye, improve reliability
• 90d0a389 - test_urls: Debug "Invalid Retry-After" error
• d302c1a2 - test_urls: Hack around malformed Retry-After on wiki.gnome.org

Compare with previous version

added 1 commit

• c1503485 - test_urls: Hack around malformed Retry-After on wiki.gnome.org

Compare with previous version

added 4 commits

• 5e1d89cf - test_urls: Print exceptions when validating URLs
• fd2d9a15 - test_urls: Always print the HTTP status code fore each URL
• 1068c049 - test_urls: Use a session to retry requests
• ec255481 - test_urls: Hack around malformed Retry-After on wiki.gnome.org

Compare with previous version

added 7 commits

• 6df9ff6e - test_urls: Use a threadpool for the HTTP requests
• 344f3ef3 - test_urls: Summarize the broken URLs at the end
• c4984102 - test_urls: Use a set to avoid O(N) insertion
• 97d04f68 - test_urls: Print exceptions when validating URLs
• 6f2f37b2 - test_urls: Always print the HTTP status code fore each URL
• b6590d9c - test_urls: Use a session to retry requests
• c7ff48b7 - test_urls: Hack around malformed Retry-After on wiki.gnome.org

Compare with previous version

added 5 commits

• 76dda750 - test_urls: Print exceptions when validating URLs
• 2d6990c6 - test_urls: Always print the HTTP status code fore each URL
• 62166ab7 - test_urls: Use a session to retry requests
• 72c89dc6 - test_urls: Hack around malformed Retry-After on wiki.gnome.org
• 6bd59b81 - test_urls: Hack around the pool getting closed

Compare with previous version