September 22, 2011

Content Syndicator

CyberSEO Pro Syndicator

The CyberSEO plugin is able to syndicate almost any content source.

RSS/Atom, XML, JSON feeds and HTML documents

CyberSEO Pro: RSS/Atom, XML, JSON feeds and HTML documents

CSV files and raw text dumps. Here you have to select the dump format and the desired post structure

CyberSEO Pro: CSV files and raw text dumps. Here you have to select the dump format and the desired post structure

To syndicate a new content source, choose the appropriate window (feed or CSV), enter the feed’s URL, paste the CSV/text dump or its URL and submit it to CyberSEO. Right after that you will be able to adjust the content source settings. All supported sources are processed as XML feeds (even raw text dumps), so they have the same options.

Basic Feed Settings

CyberSEO Pro: Basic Feed Settings

Feed Title

You can modify/assign the feed title, which will be shown in the list at the main CyberSEO Pro Syndicator page.

Feed URL

Here you can see the actual URL of the feed or an unique ID in case of CSV file.

Parse WordPress archives

If enabled and if the syndicating RSS feed was generated by another WordPress blog, CyberSEO will pull all the published posts from that blog, but not only those that are available in that feed. The plugin will parse through the WordPress archives, discover and import every single post available there – doesn’t matter now many years the blog runs and how many posts it contains. Just enable the “Parse WordPress archives” and CyberSEO will grab the whole blog at once or do it post-by-post every given time interval. In other words, if some target WordPress blog has 1,000 posts on the board and there are only 10 recent ones in its RSS feed, the CyberSEO plugin will import all 1,000.

This is an unique and very powerful function which is beyond the possibilities of other popular content automation scripts. So please use it wisely and do not import copyrighted content form other blogs without their owner’s permission! Otherwise you may face legal problems.

Shuffle feed items

Enable this option to shuffle the XML/RSS contents to make the posts be added in a random order. This function requires additional server resources, so use it wisely.

Check for duplicate posts by

Use this option to define the way CyberSEO will use to detect and ignore the duplicated posts:

    • GUID and title – the post will not be added in case if either its GUID or title match one of the existing posts.
    • GUID only – the post will not be added in case if its GUID match one of the existing posts.
    • Title only – the post will not be added in case if its title match one of the existing posts.

Post type

Select the desired post type such as “post”, “page”, “attachment”, “revision”, “nav_menu_item”, “custom_css”, “customize_changeset”, “user_request” or any defined custom post type.

Custom taxonomies

Assign custom taxonomies to a selected post type. All taxonomy tags must be devided with commas. Advanced users may also do it by means of PHP code. E.g.:

$post['custom_taxonomies']['my_custom_taxonomy'] = 'tag1, tag2, tag3';

In the example above, my_custom_taxonomy must be defined for the selected post type. I would suggest use to install the Custom Post Type UI plugin, which allows one to easily manage custom post types and custom taxonomies.

Post format

Select the desired post format “default”, “aside”, “gallery”, “link”, “image”, “quote”, “status”, “audio”, “video” or “chat”.

Attribute all posts to the following user

This option allows you to choose the author (registered WordPress user) for all posts that will be syndicated from the selected XML/RSS feed.

Syndicate this feed to the following categories

Here you can select the existing pre-created WordPress categories you want assign all the syndicated posts to.

Undefined categories

This option defines what the XML/RSS syndicator must do with the post categories if they are pre-defined in the WordPress settings. Here you can choose one of the four actions:

  • Use XML syndicator default settings – the default XML syndicator rules will be applied. You can assign these rules if you click “Alter default settings” button on the main “XML Syndicator” page.
  • Post to default WordPress category – the syndicating post will be assigned to the default category. No additional categories will be created.
  • Create new categories defined in syndicating post – all previously undefined categories, specified by the <categories> tag of the syndicating post, will be automatically created.
  • Do not syndicate post that doesn’t match at least one category defined above – if the syndicating post doesn’t meet at least one of the predefined categories, it will not be added to blog.

Create tags from category names

When this option is enabled, the CyberSEO plugin will automatically create WordPress tags from the categories, specified by the <categories> tag of the syndicating post.

Extract full articles

When this mode is enabled, the CyberSEO plugin will try to automatically extract full-text articles from the shortened RSS feed. You can freely downloaded the full-text-rss script from here or from here, unzip it and upload the files into the /wp-content/ or /wp-content/plugins/cyberseo/ folder. After that the full text extraction feature will be enabled and the “Extract full articles” option will become available.

Important: If the plugin will not be able to retrieve the full-text article, the post won’t be added.


Use Google, Yandex or DeepL service to automatically translate your posts to/from over than 100 languages.

Article Forge

Since version 8.007, CyberSEO is integrated with a powerful Article Forge service, which automatically writes completely unique, on-topic, high-quality articles using advanced artificial intelligence (AI) and deep learning. You may use it to generate a whole content of your posts, or use it to enrich the posts syndicated from RSS feeds or other sources. E.g. you can add the AI generated articles above or below the main content of your posts. Just give it a keyword and Article Forge will generate an unique text for it.

The CyberSEO plugin allows you to define a desired keyword for every syndicated post, to set its position in the main post content. Also it allows you to use the AI generated article titles instead of the original ones. Note that post duplication check settings will be ignored if the “Replace the post content” option is selected.

Important: Article Forge is a 3rd-party service, so the content generation is being performed at remote servers. So it may take a rather long time to generate an article. So I suggest you to not add more than one article at once. Also you should increase your maximum PHP execution time up to 3 minutes. To do so, just alter the max_execution_time parameter in your php.ini as follows and restart your server:

max_execution_time = 180

WPML language

This option is available only if the WordPress Multilingual Plugin is installed and active. Use the “WPML language” drop-box to select a WPML language code.

Post tags

This text field allows you to create your own tags and assign them to all posts that will be syndicated from the selected feed.

Auto tags

If this option is enabled, the CyberSEO plugin will look for existing tags within your content and add them automatically.

Check this feed for updates every […] minutes

Use this option to set the time interval in minutes which defines on how often CyberSEO should check the selected feed for updates. If you don’t want to check the feed for updates automatically, simple set this value to 0.

Maximum number of posts to be syndicated from each feed at once

Here you can set the maximum number of posts that can be syndicated from the selected feed, every time when it’s checked by plugin for the updates. It is recommended to use low values to avoid the server overload. Also it will make your self-populating site look like an organically growing one. Search engines don’t like those “blogs” that add 100 new posts at once…

Posts status

This drop-down menu allows you to define the status of syndicated posts. There are four options that barely need any additional description:

  • Hold for review
  • Save as draft
  • Save as private

Republish existing posts

If enabled, the existing posts will be republished every time the feed is pulled. Do not enable this option if you are not sure on what you are doing!

Here you can select the post status, which can be one of the following: “Publish immediately”, “Hold for review”, “Save as draft” or “Save as private”.


Use this option to enable or disable comments on the syndicating posts.


This option defines the ping status for the syndicated posts. You may either allow pings or forbid them.

Embed video

When enabled, the contents of the video custom field will be embedded into the post. The custom filed may contain a direct link to a video file (.mp4, .webm ect) or an embed code (<iframe>). The following options are available:

  • Do not embed video – the attachment will be ignored and won’t be inserted into the post.
  • Embed video at the top of the post – the video player will be placed at the top of the post.
  • Embed video at the bottom of the post – the video player will be placed at the bottom of the post.

Media attachments

Some RSS feeds have so-called media attachments (enclosures) (usually images, videos and audios) enclosed within <media:content>, <media:thumbnail> and <enclosure> (only the “image” type is supported) tags. The drop-down menu let’s you decide on what to do with these types of attachments:

  • Do not insert attachments – the attachment will be ignored and won’t be inserted into the post.
  • Insert attachments at the top of the post – the attachment will be placed at the top of the post.
  • Insert attachments at the bottom of the post – the attachment will be placed at the bottom of the post.

In case if you decide to insert the attachments, they will will contain in the <img> tag, so you will be able assign them some custom style.

Post links handling

Sometimes you may want to automatically remove a certain type of links from the syndicating posts. This option allows you to do one of the following operations:

  • Keep links intact
  • Remove all links
  • Remove all links except links in images
  • Remove links from images only

Post thumbnails

When “Generate from the first post image” option selected, the CyberSEO plugin will generate post thumbnails (also known as featured images) using one of the following sources:

  • first image found in the post content or excerpt;
  • last image found in the post content or excerpt;
  • random image found in the post content or excerpt;
  • media attachment thumbnail;
  • contents of the “thumb” custom field.

Important: if the plugin will not be able to create post thumbnail (e.g. the source image is missing or broken), the post will be deleted. So if you are going to enable the post thumbnail generation, please make sure that images are available in your content source.

Post thumbnail is required

If the plugin will not be able to create post thumbnail as specified above (e.g. the source image is missing or broken), the post will be deleted.

Shuffle paragraphs

Enable this option if you want to shuffle paragraphs of the syndicating posts.

UTF-8 encoding

This option converts an ISO-8859-1 string into UTF-8 that may be required when parsing the XML/RSS feeds containing illegal UTF-8 start bytes (e.g. <0x92>). Use it to aggregate even invalid feeds, that can not be parsed by other XML/RSS syndicators.

Convert character encoding

Enable this function if the selected feed contains is being delivered in some national character set (e.g. windows-1251) to automatically convert it into UTF-8.

Sanitize content

When enabled, the post content will be sanitized for allowed HTML tags for post content.

Store images locally

If enabled, all images from the syndicating feeds will be copied into the default uploads folder of your site. Make sure that your “/wp-content/uploads” folder is writable. This my speed up the site loading time and also will let you to syndicate the posts containing hotlink-protected images.

Store videos locally

If enabled, all videos from the syndicating feeds will be copied into the default uploads folder of your site. Make sure that your “/wp-content/uploads” folder is writable. This my speed up the site loading time. Note that this operation may take some time, so the video content may not be immediately accessible. The following video file extensions are supported: mp4, m4v, mov, wmv, avi, mpg, ogv, 3gp, 3g2.

Shorten post excerpts

Set the max number of words to be left in the post excerpts. Use 0 to remove the excerpts completely or leave it blank to keep the excerpts untouched.

Base date

Here you can choose the base date for the syndicating posts. If you select “Get date from post”, all new posts will be added with the same dates as specified in the RSS/XML feeds (in case if the dates are really specified there). But if you select “Use syndication date”, the new posts will be stamped with the actual date of their syndication.

Post date adjustment range

Here you can set the syndication date adjustment range in minutes.
This range will be used to randomly adjust the publication date for every aggregated post. For example, if you set the adjustment range as [-60] .. [60], the post dates will be randomly spread between -60 and +60 minutes.

Post lifetime

The period of time in hours after which the post will be deleted. If you don’t want to limit the post lifetime, set this parameter to 0. This feature is very useful for online shops and similar projects that need to be constantly updated.

Advanced Feed Settings

CyberSEO Pro: Advanced Feed Settings

Proxy mode

This option allows one to enable or disable proxies from the list, specified in General Settings. The proxies are usually used to parse those services that do not allow frequent connections from the same IP’s.

User agent

Use this field to simulate a user agent. E.g. a Web browser, a search engine crawler etc.

HTTP referrer

Set URL for the referer spoofing. When set to “self”, the actual feed’s URL will be used.

HTTP headers

HTTP headers from a request follow the same basic structure of an HTTP header: a case-insensitive string followed by a colon (‘:’) and a value whose structure depends upon the header. One header per line.

XML section names (separate with commas)

Normally CyberSEO Pro automatically recognizes the internal structure of XML and JSON feeds, but sometimes the feed can be a very complicated one and it may contain a few different blocks of items, so you may want to define which exactly block you want to import. As you may know, the content of every RSS entry is enclosed within the <item> tags. E.g.:

    <title>Example entry</title>
    <description>Here is some text containing an interesting description.</description>
    <guid>unique string per item</guid>
    <pubDate>Mon, 06 Sep 2009 16:45:00 +0000 </pubDate>

Thus any content automation plugin is able to properly recognize and parse the RSS feeds, since they have a very formalized structure. But if there is a need to parse an XML or JSON feed of an arbitrary format, you may face a serious problem because this task can’t be done by other content automation plugins. Fortunately this is not a problem for CyberSEO which will do it automatically. However you can always override its behavior using the “XML section tag names” field.

Let’s consider we need to parse some XML feed which entries are enclosed within the <product> tag and have the following structure:

    <name>DuroStar DS7200Q Remote Control Silent Diesel Generator</name>
    <specification><img url=""><br />Perfect for the job site or the residential user who needs back-up power Features: Electric start on panel plus remote start. Advanced direct fuel injection system for low fuel consumption. Dependable, maintenance free alternator with automatic voltage regulator. Full power panel with keyed ignition, oil alert automatic shutdown for low oil pressure, volt meter, circuit breakers, low fuel indicator and power outlets. 12 hours Continuous Operating Capability. Fully protected 12V, 120V &amp; 220V outlets. Ground Fault Interrupter. DC 12V charging system.

The CyberSEO plugin will parse it automatically, but let’s pretend it was unable to recognize the <product> enclosing tag. In this case we need to go to CyberSEO->XML Syndicator and click the “Alter default settings” button. Now we have to locate the “XML section tag names” field, add there the new enclosure tag (in the case above it’s PRODUCT) and click “Update Default Settings”. Done! Now the CyberSEO plugin will be able to recognize and parse any XML feed where all items are enclosed within the <product> tag.

However we still can’t pull this feed to the blog without some additional settings, because such important tags as <title> and <description> are missing there. Furthermore, there is an additional tag <price>, which we may also like to include into the post.

To solve this problem, we’ll use templates. First of all, we have to find the “Post title” box and alter its contents as follows:


This will tell CyberSEO to use the XML value “name” as a WordPress post/page title.

Now let’s format the body of our post. Find the “Post content” box and put there the HTML template of the post you want to be generated. E.g.:

<p><strong>Price:</strong> %xml_tags[price]%</p>

Now we just need to click “Update Feed Settings” and voila – we are ready to automatically parse this XML feed as a source of new posts for our blog!

Disable JSON section name detection

The “Disable JSON section name detection” option allows you to disable the built-in sophisticated detection of JSON section names. And forces to plugin to use only the section names, specified in the “XML section names” box. How can it be useful? Let’s take a look at the iTunes JSON feed:

It has the following structure:

    "feed": {
        "title": "Coming Soon",
        "id": "",
        "author": {
            "name": "iTunes Store",
            "uri": ""
        "links": [{
                "self": ""
            }, {
                "alternate": "\u0026app=music"
        "copyright": "Copyright © 2018 Apple Inc. All rights reserved.",
        "country": "us",
        "icon": "",
        "updated": "2020-10-25T02:16:04.000-07:00",
        "results": [{
                "artistName": "Miley Cyrus",
                "id": "1536966025",
                "releaseDate": "2020-11-27",
                "name": "Plastic Hearts",
                "kind": "album",
                "copyright": "℗ 2020 RCA Records, a division of Sony Music Entertainment",
                "artistId": "137057909",
                "contentAdvisoryRating": "Explicit",
                "artistUrl": "",
                "artworkUrl100": "",
                "genres": [{
                        "genreId": "14",
                        "name": "Pop",
                        "url": ""
                    }, {
                        "genreId": "34",
                        "name": "Music",
                        "url": ""
                "url": ""
            }, {
                "artistName": "The White Stripes",
                "id": "1534387685",
                "releaseDate": "2020-12-04",
                "name": "The White Stripes Greatest Hits",
                "kind": "album",
                "copyright": "℗ 2020 Jack White and Meg White",
                "artistId": "2456318",
                "artistUrl": "",
                "artworkUrl100": "",
                "genres": [{
                        "genreId": "21",
                        "name": "Rock",
                        "url": ""
                    }, {
                        "genreId": "34",
                        "name": "Music",
                        "url": ""
                "url": ""

If we let CyberSEO Pro to automatically detect section names, it will take the first one, which will be “<links></links>”. So we won’t be able to get the necessary information about the music albums. To solve this problem, we need to explicitly set the section we want to parse. This can be easily done as shown below.

  1. Add RESULTS_ITEM tag to the XML section names.
  2. Check (enable) the Disable JSON section name detection check box.

That’s all we need to do.

Synonymizer mode

Here you can select the synonymized (the built-in or a 3rd-party one) which will be used to alter the imported content.

Post title, Post content and Post excerpt

CyberSEO Pro allows you to define HTML templates for post title, post content and post excerpt. You can use these text boxes to define the layout and contents of the posts, generated by the plugin. The following special keywords are available:

  • %post_title% – post title;
  • %post_content% – post content;
  • %post_excerpt% – post excerpt;
  • %post_excerpt_notags% – post excerpt with stripped HTML tags;
  • %link% – a link to source;
  • %post_guid% – post GUID;
  • %media_description% – post media description (if media attachments are included);
  • %enclosure_url% – the enclosure URL (if attachment enclosure is included);
  • %xml_tags[name]% – an XML tag value, where name must be replaced with the actual XML tag name;
  • %params[name]% – a value of the specified feed URL parameter. E.g. if the feed URL is, the %params[_nkw]% will be replaced with “books”.
  • %custom_fields[name]% – a custom field value, where name must be replaced with the actual custom field name;
  • %custom_fields_attr[name][attr]% – a custom field attribute, where name must be replaced by the actual custom field name and attr must be replaced by the XML attribute name;
  • %media_thumbnail[n]% – a link to the media thumbnail, where n is its integer index (if media attachments are included).

E.g. if you want to add some text to every post title (say “Breaking news:”), you should alter the “Post title” template like this:

Breaking news: %post_title%

Here is another example. Let’s say you are going to import a product XML feed which has the following items:

<product_name>the product name here</product_name>
<picture>the product picture URL here</picture>
<price>the product price here</price>

The only thing you have to do is to simply define your “Post template” like this:

<image src="%xml_tags[picture]%" />
<p><b>Price:</b> %xml_tags[price]%</p>

Don’t synonymize titles

Use this option to allow or disallow synonymizing of the post titles.

Custom Fields

Here you can assign XML tag values to custom fields of the syndicating post, using the following format:


tag – an XML tag name you want to select. This parameter is case insensitive;
name – a name of WordPress post custom field to write the selected value to. This parameter is case sensitive.

The example below stores the values of <name> and <isbn> tags into the “book_title” and “book_isbn” custom fields:


You can also write a fixed arbitrary string value into a post custom field. To do so, you have to specify the sting value enclosed within double quotes and use it instead of the XML tag name. For example, this is how you can assign the “This is my product” string value to the “product” custom field:

"This is my product"->product

Post content shortcodes

In CyberSEO Pro you can use the cyberseo_gallery shortcode to insert an image gallery into your post. E.g.:

[cyberseo_gallery url="%link%" featured_image="first" link="file"]

The following parameters are available for the cyberseo_gallery shortcode:

  • url: the source gallery URL;
  • featured_image: use “first”, “last”, “random” or leave it empty if you don’t want to generate featured image at all;
  • max_images: max number of images in the gallery (default: all images);
  • cols: a number of columns in the gallery (default: 4);
  • size: specify the image size to use for the thumbnail display. Valid values include “thumbnail”, “medium”, “large”, “full” and any other additional image size that was registered with add_image_size(). The default value is “thumbnail”. The size of the images for “thumbnail”, “medium” and “large” can be configured in WordPress admin panel under Settings > Media.
  • link: can be “file” (link directly to image file), “none” (no link) or leave it empty if you want to link it to the attachment’s permalink.
  • on_fail: use the “delete” value to prevent import of posts which have no galleries.

Important: if you are using the cyberseo_gallery shortcode, the “Post thumbnail” option must be set to “Do not generate” otherwise it will be ignored.

Post headers and post footers

Everything that you put into these boxes will be added to every syndicated post at it’s top or bottom accordingly. You can use the post headers and footers to add some random text or HTML code to every syndicating post. In this case, each chunk of text must be separated with the “<!–more–>” marker. For example, if you put the following text into the “Post Footers” box, every syndicated post will be amplified with a random Mark Twain’s quote:

I was gratified to be able to answer promptly. I said I don't know.
If you pick up a starving dog and make him prosperous, he will not bite you. This is the principal difference between a dog and a man.
If you tell the truth you don't have to remember anything.
In Paris they simply stared when I spoke to them in French; I never did succeed in making those idiots understand their language.
In religion and politics, people's beliefs and convictions are in almost every case gotten at second hand, and without examination.
In the first place, God made idiots. That was for practice. Then he made school boards.
It could probably be shown by facts and figures that there is no distinctly American criminal class except Congress.
It is easier to stay out than get out.

You can also use the predefined tags (see Post Title, Post Content and Post Excerpt), which available for post title, post content and post excerpt templates. For example, if you want to include a link to source at the bottom of every generated post, you can do it by inserting the following line into the “Post footers” box:

<p><a href="%link%">Source</a></p>

PHP code <?php .. ?>

This is the most powerful tool which is intended for advanced users who are familiar with PHP. Use it to execute your own PHP code for every aggregating post, which info is represented as the $post array variable. You can alter this variable in order to apply your own changes to every aggregating post. The following items are defined in the $post array variable:

  • $post[‘post_title’] – post title;
  • $post[‘post_name’] – post name;
  • $post[‘post_type’] – post type;
  • $post[‘link’] – link;
  • $post[‘post_content’] – post content;
  • $post[‘post_excerpt’] – post excerpt;
  • $post[‘guid’] – GUID;
  • $post[‘post_author’] – post author;
  • $post[‘post_date’] – post date in Unix timestamp format;
  • $post[‘categories’] – array of post category names;
  • $post[‘media_content’] – array of media content URLs;
  • $post[‘media_thumbnail’] – an array of media thumbnail URLs;
  • $post[‘enclosure_url’] – enclosure URL;
  • $post[‘enclosure_type’] – type of enclosure URL. E.g.: audio/mpeg;
  • $post[‘custom_fields’] – array of custom fields;
  • $post[‘custom_fields_attr’] – array of XML tag attributes for corresponding custom fields;
  • $post[‘comments’] – the array may contain post comments in WordPress format.
  • $post[‘custom_fields’] – array of post custom fields;
  • $post[‘custom_fields_attr’] – array of XML attributes associated with the post custom fields;
  • $post[‘tags_input’] – an empty array. You can use it to assign additional tags to syndicating posts via the $post[‘tags_input’] variable. E.g.:
    $post['tags_input'] = array('tag1', 'tag2', 'tag3', 'etc');

If you modify the $post array (say change the post tittle, insert some text into the post’s body, or create your own tags according to the post’s content), your changes will be applied to the post BEFORE it will be processed by CyberSEO and added to the blog. If you don’t want some particular post to be syndicated (say it contains some undesirable words etc), your PHP code must assign FALSE to the $post variable.


Make all post titles uppercase:

$post['post_title'] = ucwords(mb_strtolower($post['post_title']));

Add 12 hours to the post dates:

$post['post_date'] += 12 * 60 * 60;

Copy post content into post excerpt:

$post['post_excerpt'] = $post['post_content'];

Assign a random value between 1 and 100 (inclusive) to a custom field named “probability”:

$post['custom_fields']['probability'] = rand(1, 100);

Limit post content to 30 words, excerpt to 25 words and add a link to source article below:

$max_content = 30;
$max_excerpt = 25;
$words = explode(" ", trim($post['post_content']));
if (count($words) > $max_content) {
$post['post_content'] = implode(" ", array_slice($words, 0, $max_content)) . '<br /><a href="' . $post['link'] . '">Source...</a>';
$words = explode(" ", trim($post['$max_excerpt']));
if (count($words) > $max_excerpt) {
$post['post_excerpt'] = implode(" ", array_slice($words, 0, $max_excerpt)) . '<br /><a href="' . $post['link'] . '">Source.</a>';

Syndicate ONLY the posts that contain word1, word2 or word3:

if (!preg_match ('/word1|word2|word3/is', $post['post_title'] . $post['post_content'] . $post['post_excerpt'])) {
    $post = false;

Do NOT syndicate the posts that contain word1, word2 or word3:

if (preg_match ('/word1|word2|word3/is', $post['post_title'] . $post['post_content'] . $post['post_excerpt'])) {
    $post = false;

Content filtering Settings

CyberSEO Pro: Content filtering Settings
This section of the feed settings page allows you to filter syndicated content by keywords and phrases (separate them with commas), tags, publication date and text lenght. The foolowing filters are available:

  • Must contain any of these keywords […]
  • Must contain all these keywords […]
  • Must contain none of these keywords […]
  • Must contain any of these tags […]
  • Must contain none of these tags […]
  • Must be newer than […] days.
  • Must be older than […] days.
  • Must be longer than […] characters.