[Ingest] Route scraping through an optional V2Ray/Xray proxy (Telegram in Iran)

Telegram and some sources are filtered in Iran. .NET cannot speak vmess/vless/trojan, so add an Xray sidecar (compose service 'xray', behind the 'proxy' profile) that converts the admin's config into a local SOCKS5 proxy (xray:10808). New ScrapeHttpClients provider builds a proxied or direct HttpClient (WebProxy supports socks5/socks4/http) cached per proxy URL; all five ingestion sources (Telegram/Bale/Divar/Medjobs/Websites) now use it. Admin settings gain IngestProxyEnabled + IngestProxyUrl (migration; UI under sources). Added deploy/xray/config.json template + README with vmess/vless/trojan examples. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:53:17 +03:30
parent 698565c460
commit cea27c8684
17 changed files with 1411 additions and 20 deletions
@@ -37,6 +37,8 @@ builder.Services.AddHttpClient("scrape", c =>
    c.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (compatible; HamkadrBot/1.0)");
 });
 builder.Services.AddHttpClient("ai");
+// Proxy-aware client provider for ingestion (routes through Xray/V2Ray SOCKS proxy when set).
+builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.ScrapeHttpClients>();
 builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.ListingValidator>();
 builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.IAiAuditor,
    JobsMedical.Web.Services.Scraping.OpenAiCompatibleAuditor>();