33 Commits

Author	SHA1	Message	Date
Egor Aristov	271c1fac48	small fixes	2025-05-07 03:54:23 +03:00
Egor Aristov	129e95f556	extract from attrubite	2025-05-07 03:54:19 +03:00
Egor Aristov	9a55cebb50	update adblock files	2025-04-29 11:29:51 +03:00
Egor Aristov	e0af3770f6	adblock	2025-04-24 22:20:37 +03:00
Egor Aristov	31bbc97f9b	refactoring	2025-03-23 09:35:08 +03:00
Egor Aristov	91475c2c4d	refactoring	2025-03-23 09:11:17 +03:00
Egor Aristov	1e42eb81ea	retries	2025-03-20 22:43:36 +03:00
Egor Aristov	a0876409c5	retries	2025-03-20 22:27:36 +03:00
Egor Aristov	87f7c017de	increase screenshot height	2025-03-18 14:04:35 +03:00
Egor Aristov	45bb528917	removed waiting for 1st post (on some sites "1 post is fully rendered" != "whole page loaded")	2025-03-18 14:04:32 +03:00
Egor Aristov	b9f77388a1	unit tests...	2025-03-18 14:03:52 +03:00
Egor Aristov	220bc50e47	block local ips and service worker; util function to resolve arbitrary 'host-like' string, unit testing	2025-03-18 14:03:52 +03:00
Egor Aristov	08326debdd	block local ips and service worker; util function to resolve arbitrary 'host-like' string, unit testing	2025-03-18 14:03:52 +03:00
Egor Aristov	8d21583f9e	prevent webrtc leak through proxy	2025-03-18 14:03:52 +03:00
Egor Aristov	83ffba44f7	log visit time to "info" instead of "debug"	2025-03-18 14:03:51 +03:00
Egor Aristov	bca61c5bd3	opt-in to "new headless"	2025-03-18 14:03:51 +03:00
Egor Aristov	5aba78ed66	by domain rate limiter	2025-03-18 14:03:51 +03:00
Egor Aristov	72ec41dccf	extractor rewrite (use custom wrappers over playwright locator)	2025-03-18 14:03:50 +03:00
Egor Aristov	0d7c238e65	fix: skip posts with empty date	2025-03-18 14:02:36 +03:00
Egor Aristov	da66b8de73	log cookie header	2025-03-18 14:02:36 +03:00
Egor Aristov	f956d883bf	change timeouts	2025-03-18 14:02:36 +03:00
Egor Aristov	6c51d7b24b	cookie manager	2025-03-18 14:02:35 +03:00
Egor Aristov	925bf49a8e	compare revs (for testing) ... Support mocking dates Support mocking dates Support mocking dates Support mocking dates	2025-03-18 14:02:34 +03:00
Egor Aristov	0d48fe8554	date parsing enhancements	2025-03-18 14:01:43 +03:00
Egor Aristov	f78345ab14	allow empty description or content	2025-03-18 14:01:42 +03:00
Egor Aristov	ef7433a594	cookies fix	2025-03-18 14:01:41 +03:00
Egor Aristov	fed51635f5	headers and cookies support	2025-03-18 14:01:41 +03:00
Egor Aristov	bd33314298	screenshots; some refactoring	2025-03-18 14:01:41 +03:00
Egor Aristov	e3ad088dbd	change consumer to durable	2025-03-18 14:01:41 +03:00
Egor Aristov	155cb37735	speed up loading by waiting for EITHER locator OR networkIdle	2025-03-18 14:01:40 +03:00
Egor Aristov	94694b2fee	proxy support; config validation	2025-03-18 14:01:40 +03:00
Egor Aristov	1cfc3ca0a7	extractor (refactoring and enhancements)	2025-03-18 14:01:39 +03:00
Egor Aristov	e80de33083	initial commit	2025-03-18 14:01:39 +03:00