Implement HTML domain checker in Rust and add PowerShell script for chunked execution

- Added a Rust program that reads a list of domains from "domains.txt", checks if they return HTML content, and writes valid domains to "domains-filtered.txt".
- Introduced a PowerShell script to execute a Python script in chunks, allowing for processing of specified line ranges from the domain list.
This commit is contained in:
imnyang 2025-06-06 23:47:59 +09:00
commit 351af7ba78
9 changed files with 36159 additions and 6 deletions

8
is-html-fast/Cargo.toml Normal file
View file

@ -0,0 +1,8 @@
[package]
name = "is-html-fast"
version = "0.1.0"
edition = "2024"
[dependencies]
rayon = "1.10.0"
reqwest = { version = "0.12.19", features = ["blocking", "json"]}