26 Commits

Author	SHA1	Message	Date
Egor Aristov	1b91f55fd0	adblock	2025-03-22 11:54:16 +03:00
Egor Aristov	1e42eb81ea	retries	2025-03-20 22:43:36 +03:00
Egor Aristov	a0876409c5	retries	2025-03-20 22:27:36 +03:00
Egor Aristov	87f7c017de	increase screenshot height	2025-03-18 14:04:35 +03:00
Egor Aristov	220bc50e47	block local ips and service worker; util function to resolve arbitrary 'host-like' string, unit testing	2025-03-18 14:03:52 +03:00
Egor Aristov	08326debdd	block local ips and service worker; util function to resolve arbitrary 'host-like' string, unit testing	2025-03-18 14:03:52 +03:00
Egor Aristov	8d21583f9e	prevent webrtc leak through proxy	2025-03-18 14:03:52 +03:00
Egor Aristov	83ffba44f7	log visit time to "info" instead of "debug"	2025-03-18 14:03:51 +03:00
Egor Aristov	bca61c5bd3	opt-in to "new headless"	2025-03-18 14:03:51 +03:00
Egor Aristov	5aba78ed66	by domain rate limiter	2025-03-18 14:03:51 +03:00
Egor Aristov	72ec41dccf	extractor rewrite (use custom wrappers over playwright locator)	2025-03-18 14:03:50 +03:00
Egor Aristov	0d7c238e65	fix: skip posts with empty date	2025-03-18 14:02:36 +03:00
Egor Aristov	da66b8de73	log cookie header	2025-03-18 14:02:36 +03:00
Egor Aristov	f956d883bf	change timeouts	2025-03-18 14:02:36 +03:00
Egor Aristov	6c51d7b24b	cookie manager	2025-03-18 14:02:35 +03:00
Egor Aristov	925bf49a8e	compare revs (for testing) ... Support mocking dates Support mocking dates Support mocking dates Support mocking dates	2025-03-18 14:02:34 +03:00
Egor Aristov	0d48fe8554	date parsing enhancements	2025-03-18 14:01:43 +03:00
Egor Aristov	f78345ab14	allow empty description or content	2025-03-18 14:01:42 +03:00
Egor Aristov	ef7433a594	cookies fix	2025-03-18 14:01:41 +03:00
Egor Aristov	fed51635f5	headers and cookies support	2025-03-18 14:01:41 +03:00
Egor Aristov	bd33314298	screenshots; some refactoring	2025-03-18 14:01:41 +03:00
Egor Aristov	e3ad088dbd	change consumer to durable	2025-03-18 14:01:41 +03:00
Egor Aristov	155cb37735	speed up loading by waiting for EITHER locator OR networkIdle	2025-03-18 14:01:40 +03:00
Egor Aristov	94694b2fee	proxy support; config validation	2025-03-18 14:01:40 +03:00
Egor Aristov	1cfc3ca0a7	extractor (refactoring and enhancements)	2025-03-18 14:01:39 +03:00
Egor Aristov	e80de33083	initial commit	2025-03-18 14:01:39 +03:00