Log In
Home
(0 votes) 1:58 pm
March 3, 2024
Offline
Hii Cyberseo
Currently, CyberSEO may re-process the same URL multiple times, leading to inefficient use of resources and potentially redundant actions (full text extractor, etc). This can occur especially when filters are in place, example skip to process url if content contain xxxx.
This idea to prevent redundant url processing.
CyberSEO maybe can logs processed URLs, it can be small TXT file (store last 100 url or 1000).
This would allow the system to check against this list before processing a URL, ensuring each unique URL is handled only once.
3:15 pm
July 2, 2009
Offline
What do you mean by "processing the same URL multiple times"? If you're referring to different feeds importing the same URL independently, that's expected behavior. If you mean multiple processing of the same post within a single feed - please provide a log example. The plugin logs each cURL request step-by-step, so if that's happening, it should be visible.
As for storing processed URLs in a separate file - that's redundant. The plugin already checks for duplicates and applies filters during processing. An external TXT-based log wouldn't improve this process and might even slow it down.
7:10 pm
March 3, 2024
Offline
Example like this:
[01-04-25 05:03:41] Processing a new post: 
Login to see this link
[01-04-25 05:03:41] Checking for duplicate by link
[01-04-25 05:03:42] Trying to extract full text article with Full-Text RSS script
[01-04-25 05:03:49] Done
[01-04-25 05:03:49] Apply post filtering
[01-04-25 05:03:49] The post is too short
[01-04-25 05:03:49] The post will not be added
and when syndicate run again after 10-12 hours
[01-04-25 15:07:07] Processing a new post: Login to see this link
[01-04-25 15:07:07] Checking for duplicate by link
[01-04-25 15:07:08] Trying to extract full text article with Full-Text RSS script
[01-04-25 15:07:09] Done
[01-04-25 15:07:09] Apply post filtering
[01-04-25 15:07:09] The post is too short
[01-04-25 15:07:09] The post will not be added
cyberseo will extract full text article with Full-Text RSS script again, apply post filtering again (count characters).
All RSSMost Users Ever Online: 541
Currently Online:
14 Guest(s)
Currently Browsing this Page:
1 Guest(s)
Top Posters:
ninja321: 86
s.baryshev.aoasp: 68
harboot: 61
Freedom: 61
Pandermos: 54
MediFormatica: 49
Member Stats:
Guest Posters: 337
Members: 2980
Moderators: 0
Admins: 1
Forum Stats:
Groups: 1
Forums: 5
Topics: 1697
Posts: 8678
Newest Members:
raffaellabonaschi, matogon, nduwawep, burakaltanalisan, info.magicbytes, ioannis.mavroudisAdministrators: CyberSEO: 4090