The Problem We have a raw list of 3,000 root domains (e.g., `example-site.com`). However, we need to find specific articles or sub-pages (e.g., `example-site.com/specific-topic-article`) that are relevant to our client's niche. The solution must be capable of taking a homepage as input, "expanding" the site structure to find deep content, and filtering out irrelevant pages (e.g., "About Us", generic news, or off-topic content). Input Data You are provided with two datasets: - 'Input_Domains.csv`: A raw export of 3,000 root domains. Note: This is "messy" real-world data containing both high-relevance sites and irrelevant "junk" sites. Target_Keywords.csv: A list of specific keywords that define "Relevance" for this project. The Task Your solution should process the inputs and execute the following pipeline: 1. Expansion & Crawling: Iterate through the domains and crawl the site to discover relevant deep pages (articles, guides, reviews). 2. Semantic Filtering: Score each identified page against the `Target_Keywords.csv` list. A page is a "Match" if it contains target keywords in its core metadata (Title, H1, URL). 3. Exclusion: Automatically discard "utility" pages (e.g., Privacy Policy, Login) and irrelevant topics. Deliverables 1. Source Code / Methodology: The complete code, script, or workflow used to process the list. 2. Final Spreadsheet (CSV): The output file generated by your solution using our provided sample data. Success Metrics - Precision: Does the output successfully exclude the irrelevant domains included in the input? - Granularity: Does the output provide specific deep URLs rather than just homepages?