Content Syndicator – CyberSEO Pro
September 22, 2011

Content Syndicator

CyberSEO Pro Syndicator

The CyberSEO Pro Syndicator is able to parse almost any content source. It supports RSS, Atom and JSON feeds, as well as CSV and raw text dumps. To syndicate a new content source, choose the appropriate window (feed or CSV), enter the feed’s URL, paste the CSV/text dump or its URL and submit it to CyberSEO. Right after that you will be able to adjust the content source settings. All supported sources are processed as XML feeds (even raw text dumps), so they have the same options.

The same menu allows you to import social media resources like Twitter, TikTok, Instagram, Telegram and Reddit, as well as links to Amazon product search pages or Sitemap XML’s. In some cases you need to select the right preset, in other cases the type of content source you want to import is recognized and processed automatically.

RSS/Atom, XML, JSON and CSV feeds, social networks, tubes, marketplaces

CyberSEO Pro Syndicator Basic

CSV files and raw text dumps.

CyberSEO Pro provides you two basic ways to import CSV files.

Method #1

In case if you want to import a CSV file from a remote host, just enter its URL into the “RSS/XML/JSON/CSV source” box and enable the “Treat as CSV ” option. This method allows you to import large dynamically updated CSV files with a header (the column names for the CSV data are defined in the line of the file). For example, it’s the right way to import large product CSV sheets from online marketplaces like AliExpress, Admitad etc. This method toes not use your server memory and database to store the source data, because the syndicating content is located at a remote host.

Method #2

This method lets you import CSV or comma/semicolon/pipe delimited raw text dumps without a header (the column names are not defined in the CSV sheet and must be manually assigned by the user). Just copy-paste the CSV content into the “CSV or text dump” box, select the delimiter type and assign the desired post element names such as title, content, description, link, thumbnail, date etc. Please remember that this method can be used with static data only. Also it uses your server memory to store all the content there. Thus it’s not recommended to use it with large product lists.

Here you have to select the dump format and the desired post structure:

CyberSEO Pro: CSV files and raw text dumps. Here you have to select the dump format and the desired post structure

Quick override default settings

Use this form to temporary modify some default CyberSEO content syndicator settings, before the feed URL will be added. Here you can set a fake user agent, post headers etc. These settings will be applied to one particular feed which will be added next. In contrast to feed’s default settings, these options won’t be stored in the database.

Mass modify selected feeds

You know how difficult it is to change a particular option or group of options for several feeds at once in your CyberSEO Pro Syndicator. This is what the “Modify selected feeds” button is intended for. Just select the feeds you want and click it. Use the red check box to the left of each feed option you want to apply it to all selected feeds. Unchecked options will not be applied.

Batch feed settings import/export

Use the menu at the bottom of the “Content Syndicator” page to batch import and export your selected feed settings from/to a text file. It makes it easy to save your feed settings and to use them on other sites.

Basic Settings

CyberSEO Pro Syndicator Basic

Preview mode

In the top left corner of the feed settings page you can see the “Preview mode” drop-box. Use it to switch the post preview mode. The following options may be available:

  • Post content
  • Full text article
  • Attachment
  • XML structure

Since CyberSEO Pro converts all content sources into XML, all importing sources can be displayed as an XML structure. The rest 3 modes maybe available or not, depending on the source content. For example, if the plugin was unable to extract the full text article, the corresponding option won’t be shown. If there is no media attachment, the “Attachment” option will be hidden. If there is no <description> or <content> XML section, the “Post content” option won’t be shown too.

Thus, the “Preview mode” drop-box allows you to examine the syndicating content source. You can see its XML structure, the attached media (if any) and you may see the difference between the original article from the feed and its full version, scrapped directly from a website to decide which exactly version of the same article you want to import.

Feed title

You can modify/assign the feed title, which will be shown in the list at the main CyberSEO Pro Syndicator page.

Feed URL

Here you can alter the actual URL of the feed or an unique ID in case of CSV file.

Parse WordPress archives

If enabled and if the syndicating RSS feed was generated by another WordPress blog, CyberSEO Pro will pull all the published posts from that blog, but not only those that are available in that feed. The plugin will parse through the WordPress archives, discover and import every single post available there – doesn’t matter now many years the blog runs and how many posts it contains. Just enable the “Parse WordPress archives” and the plugin will grab the whole blog at once or do it post-by-post every given time interval. In other words, if some target WordPress blog has 1,000 posts on the board and there are only 10 recent ones in its RSS feed, the CyberSEO Pro plugin will import all 1,000.

This is an unique and very powerful function which is beyond the possibilities of other popular content curation plugins. So please use it wisely and do not import copyrighted content form other blogs without their owner’s permission! Otherwise you may face legal problems.

Shuffle feed items

Enable this option to shuffle the XML entries (consider them as posts in the RSS feed) to make them added in a random order.

Check for duplicate posts by

Use this option to define the way CyberSEO will use to detect and ignore the duplicated posts:

  • GUID and title – the post will not be added in case if either its GUID or title match one of the existing posts.
  • GUID only – the post will not be added in case if its GUID match one of the existing posts.
  • Title only – the post will not be added in case if its title match one of the existing posts.
  • Don’t check for duplicate posts – the post uniqueness will not be checked.

Post type

Select the desired post type such as post, page, attachment, revision, nav_menu_item, custom_css, customize_changeset, user_request or any custom post type, such as WooCommerce product etc.

Custom post template

Here you can select a custom template for posts belonging to custom post types.

Custom taxonomies

Assign custom taxonomies to a selected post type. All taxonomy tags must be divided with commas.

Advanced users may also do it by means of PHP code. E.g.:

$post['custom_taxonomies']['my_custom_taxonomy'] = 'tag1, tag2, tag3';

In the example above, my_custom_taxonomy must be defined for the selected post type. We would suggest use to install the Custom Post Type UI plugin, which allows you to easily manage custom post types and custom taxonomies.

Post format

Select the desired post format default, aside, gallery, link, image, quote, status, audio, video or chat.

Post author

This option allows you to choose the author (registered WordPress user) for all posts that will be syndicated from the selected XML/RSS feed.

Categories

Here you can select the existing pre-created WordPress categories you want assign all the syndicated posts to.

Undefined categories

This option defines what the CyberSEO syndicator have to do if none of the post categories match the predefined defined ones. Here you can choose one of the four actions:

  • Use XML syndicator default settings – the default XML syndicator rules will be applied. You can assign these rules if you click “Alter default settings” button on the main “XML Syndicator” page.
  • Post to default WordPress category – the syndicating post will be assigned to the default category. No additional categories will be created.
  • Create new categories defined in syndicating post – all previously undefined categories, specified by the <categories> tag of the syndicating post, will be automatically created.
  • Do not syndicate post that doesn’t match at least one category defined above – if the syndicating post doesn’t meet at least one of the predefined categories, it will not be added to blog.

Create tags from category names

When this option is enabled, the CyberSEO Pro plugin will automatically create WordPress tags from the categories, specified by the <categories> tag of the syndicating post. But wait, why we need to do this?

Both RSS and Atom feeds have no such a thing as tag. They classify articles by category instead, while tag is a WordPress-specific thing. Yes, many other XML-based data formats use tags as well, but not RSS. Those use post categories only. Thus when you import a post from an feed, you may want to convert its RSS categories into WordPress tags. That’s why CyberSEO has a special option for that.

WPML language

This option is available only if the WordPress Multilingual plugin is installed and active. Use the “WPML language” drop-box to select a WPML language code.

Polylang language

This option is available only if the WordPress Polylang plugin is installed and active. Use the “Polylang language” drop-box to select a Polylang language code.

Post tags

This text field allows you to create your own tags and assign them to all posts that will be syndicated from the selected feed.

Auto tags

If this option is enabled, the CyberSEO Pro plugin will look for existing tags within your content and add them automatically. For example, the following tags are already manually defined at your site via Posts->Tags WordPress dashboard menu: sport, weather, toys and books. The following post content will be tagged as weather and sport:

“The weather today is very dank, which is not conducive to sport activities…”

Check this feed for updates every […] minutes

Use this option to set the time interval in minutes which defines on how often CyberSEO Pro should check the selected feeds for updates. If you don’t want to check the feed for updates automatically, simple set this value to 0.

Maximum number of posts

Here you can set the maximum number of posts that can be syndicated from the selected feed, every time when it’s checked by plugin for the updates. It is recommended to use low values to avoid the server overload. Also it will make your self-populating site look like an organically growing one. Search engines don’t like those “blogs” that add 100 new posts at once…

Re-publish existing posts

If enabled, the existing posts will be re-published every time the feed is pulled. Do not enable this option if you are not sure on what you are doing. This feature is mostly demanded by the adult affiliate webmasters that promote live cam sites. They use it to have the currently online cam rooms at the top of the main index page.

Post status

This drop-down menu allows you to define the status of syndicated posts. There are four options that barely need any additional description:

  • Hold for review
  • Save as draft
  • Save as private

Comments

Use this option to enable or disable comments on the syndicating posts.

Pings

This option defines the ping status for the syndicated posts. You may either allow pings or forbid them.

Links handling

Sometimes you may want to automatically remove a certain type of links from the syndicating posts. This option allows you to do one of the following operations:

  • Keep links intact
  • Remove all links
  • Remove all links except links in images
  • Remove links from images only

Shuffle paragraphs

Enable this option if you want to shuffle paragraphs of the syndicating posts.

UTF-8 encoding

This option converts an ISO-8859-1 string into UTF-8 that may be required when parsing the XML/RSS feeds containing illegal UTF-8 start bytes (e.g. <0x92>). Use it to aggregate even invalid feeds, that can not be parsed by other XML/RSS syndicators.

Convert character encoding

Enable this function if the selected feed contains is being delivered in some national character set (e.g. windows-1251, ISO-8859-1) to automatically convert it into UTF-8.

Sanitize content

Sanitizes the post content. When enabled, the plugin does the following:

  • checks for invalid UTF-8;
  • converts single < characters to entities;
  • strips all tags;
  • removes line breaks, tabs, and extra whitespace;
  • strips octets.

Balance tags

Balances redundant and unclosed HTML tags of the post content and the post excerpt via WordPress API.

  • This option shall only be used if the length of the post content can be controlled; otherwise memory issues or some obscure bugs can occur.
  • This option uses two hard-coded lists of elements for single tags and nestable tags. WordPress uses multiple such lists and the lists not kept in sync. If an element isn’t part of these lists or changed its nesting behavior, it may lead to incorrect markup.

Shorten post excerpts

Set the max number of words to be left in the post excerpts. Use 0 to remove the excerpts completely or leave it blank to keep the excerpts untouched.

Base date

Here you can choose the base date for the syndicating posts. If you select “Get date from post”, all new posts will be added with the same dates as specified in the RSS/XML feeds (in case if the dates are really specified there). But if you select “Use syndication date”, the new posts will be stamped with the actual date of their syndication.

Post date adjustment range

The options sets the syndication date adjustment range in minutes. This range will be used to randomly adjust the publication date for every aggregated post. For example, if you set the adjustment range as [-60] .. [60], the post dates will be randomly spread between -60 and +60 minutes.

For example if you want to scatter the syndicated posts over time to prevent them from appearing at your site simultaneously, you can set their adjustment range to say [1] .. [1440], so they will be added with a “pending” status with some random publication dates scattered over a 24-hour period.

Post lifetime

The period of time in hours after which the post will be deleted. If you don’t want to limit the post lifetime, set this parameter to 0. This feature is very useful for online shops and similar projects that need to be constantly updated.

Advanced Settings

CyberSEO Pro Syndicator Advanced

Extract full text articles

When this mode is enabled, the CyberSEO plugin will try to automatically extract full-text articles from the shortened RSS feed.

⚠ Important: If the plugin will not be able to retrieve the full-text article, the post won’t be added.

Translation

Use Google, Yandex or DeepL service to automatically translate your posts to/from over than 100 languages. You have to set the source and destination languages and enter your service API key (Google Translate and DeepL are paid and Yandex Translate is free). If translation fails, the post will not be added.

Built-in synonymizer

You can disable the built-in synonymizer, or set it to be applied before or after a 3rd-party spinner. The synonym table must be configured on the Content Spinners page.

Parse Spintax

Converts Spintax in post templates into a regular spun text.

Content spinner

Use this drop-down list to select a desired 3rd-party spinner, which must be configured on the Content Spinners page.

Templates

CyberSEO Pro allows you to define HTML templates for post title, post slug, post content and post excerpt. You can use these text boxes to define the layout and contents of the posts, generated by the plugin. The following template shortcodes are available:

    • %post_title% – post title;
    • %post_content% – post content;
    • %post_excerpt% – post excerpt;
    • %post_excerpt_notags% – post excerpt with stripped HTML tags;
    • %link% – a link to source;
    • %post_guid% – post GUID;
    • %media_description% – post media description (if media attachments are included);
    • %enclosure_url% – the enclosure URL (if attachment enclosure is included);
    • %xml_tags[name]% – an XML tag value, where name must be replaced with the actual XML tag name;
    • %params[name]% – a value of the specified feed URL parameter. E.g. if the feed URL is http://www.ebay.com/sch/i.html?_nkw=books&_rss=1, the %params[_nkw]% will be replaced with “books”.
    • %custom_fields[name]% – a custom field value, where name must be replaced with the actual custom field name;
    • %custom_fields_attr[name][attr]% – a custom field attribute, where name must be replaced by the actual custom field name and attr must be replaced by the XML attribute name;
    • %media_thumbnail[n]% – a link to the media thumbnail, where n is its integer index (if media attachments are included);
    • %random_image[keyword]% – a random image URL for a given keyword from the Google Images search results with the “Creative Commons licenses” filter applied.
    • %youtube_video[keyword]% – a YouTube video embed code for a given keyword.

You can use shortcode inside of shortcode. Foe example the following shortcode will insert a YouTube video relevant to the current post title right into its body:

%youtube_video[%post_title%]%

This one will add a relevant image:

%random_image[%post_title%]%

A cool feature, isn’t it?

If you want to add some text to every post title (say “Breaking news:”), you should alter the “Post title” template like this:

Breaking news: %post_title%

The “Post content” template below inserts a random image of a Christmas tree at the top of the post:

<img class="aligncenter" src="%random_image[christmas tree]%" />
%post_content

Don’t need a picture of Christmas tree? Not a problem, this template will a random creative commons image found in Google for the actual post title:

<img class="aligncenter" src="%random_image[%post_title%]%" alt="%post_title%" />
%post_content

This “Post content” template adds a backlink below the post content:

%post_content <p>Source: <a href="%link%" target="_blank">%link%</a></p>.

Here is another example. Let’s say you are going to import a product XML feed which has the following items:

<product_name>
    the product name
</product_name>
<picture>
    the product picture URL
</picture>
<price>
    the product price
</price>

The only thing you have to do is to simply define your “Post template” like this:

<h3>%xml_tags[product_name]%</h3>
<image src="%xml_tags[picture]%" />
<p><b>Price:</b> %xml_tags[price]%</p>

Don’t forget that CyberSEO Pro supports true nested Spintax. Thus feel free to use it in your tamplates.

Shortcodes

In addition to the simple template shortcodes described in the previous section, CyberSEO Pro also supports WordPress-style shortcodes. These WordPress-style shortcodes allow you to use various additional arguments.

It’s important to know that these shortcodes only work when the plugin generates posts automatically, and they won’t work when you manually add posts or edit them.

Also, unlike the simple template shortcodes, WordPress-style shortcodes can only be used in the “Post template”. They are not available in the “Post title”, “Post slug”, and “Post excerpt” templates.

[cyberseo_openai] shortcode

Use this the cyberseo_openai shortcode to insert OpenAI GPT-3 generated texts into your posts. The shortcode has the following format:

[cyberseo_openai prompt="your OpenAI GPT3 assignment" max_tokens="1000" temperature="0.3" top_p="1.0" best_of="1" presence_penalty="0.1" frequency_penalty="frequency_penalty"]

The prompt parameter is mandatory. It contains your assignment for OpenAI GPT-3. All other parameters are optional and can be omitted. Please see the description of these parameters here.

[cyberseo_dalle] shortcode

Use this the cyberseo_dalle shortcode to insert OpenAI DALL∙E 2 generated images into your posts. The shortcode has the following format:

[cyberseo_dalle prompt="your OpenAI DALL∙E assignment" name="%post_title%" n="1" size="512x512" class="aligncenter"]

The prompt parameter is mandatory. It contains your assignment for OpenAI DALL∙E. The name string is be used as a filename and as an image alt attribute value. All other parameters are described here.

[cyberseo_stable_diffusion] shortcode

Use this the cyberseo_stable_diffusion shortcode to insert OpenAI DALL∙E 2 generated images into your posts. The shortcode has the following format:

[cyberseo_stable_diffusion text="your Stable Diffusion assignment" name="%post_title%" stable_diffusion_engine="stable-diffusion-512-v2-1" cfg_scale="7" clip_guidance_preset="NONE" height="512" width="512" steps="50" class="aligncenter"]

The text parameter is mandatory. Itcontains your assignment for Stable Diffusion 2.1. The name string is be used as a filename and as an image alt attribute value. All other parameters are described here.

[cyberseo_gallery] shortcode

You can use the cyberseo_gallery shortcode in your post templates in order to insert an image gallery into your post. E.g.:

[cyberseo_gallery url="%link%" featured_image="first" link="file"]

The following parameters are available for the cyberseo_gallery shortcode:

  • url: the source gallery URL;
  • featured_image: use first, last, random or leave it empty if you don’t want to generate featured image at all;
  • max_images: max number of images in the gallery (default: all images);
  • cols: a number of columns in the gallery (default: 4);
  • size: specify the image size to use for the thumbnail display. Valid values include thumbnail, medium, large, full and any other additional image size that was registered with add_image_size(). The default value is thumbnail. The size of the images for thumbnail, medium and large can be configured in WordPress admin panel under Settings > Media.
  • link: can be file (links directly to image file), none (no link) or leave it empty if you want to link it to the attachment’s permalink.
  • on_fail: use the delete value to prevent import of posts which have no galleries.

⚠ Important: if you are using the cyberseo_gallery shortcode, the “Post thumbnail” option must be set to “Do not generate” otherwise it will be ignored.

Remove emojis from post slugs

Funny emojis may look nice in post titles, but almost everyone don’t want to see them in slugs (the post links). Enable this option to safely strip them off from your post slugs.

Don’t synonymize titles

Use this option to allow or disallow synonymizing of the post titles.

Post headers and post footers

Everything that you put into these boxes will be added to every syndicated post at it’s top or bottom accordingly. You can use the post headers and footers to add some random text or HTML code to every syndicating post. In this case, each chunk of text must be broken up with the <!--more--> separator srting. For example, if you put the following text into the “Post Footers” box, every syndicated post will be amplified with a random Mark Twain’s quote:

I was gratified to be able to answer promptly. I said I don’t know.
<!–more–>
If you pick up a starving dog and make him prosperous, he will not bite you. This is the principal difference between a dog and a man.
<!–more–>
If you tell the truth you don’t have to remember anything.
<!–more–>
In Paris they simply stared when I spoke to them in French; I never did succeed in making those idiots understand their language.
<!–more–>
In religion and politics, people’s beliefs and convictions are in almost every case gotten at second hand, and without examination.
<!–more–>
In the first place, God made idiots. That was for practice. Then he made school boards.
<!–more–>
It could probably be shown by facts and figures that there is no distinctly American criminal class except Congress.
<!–more–>
It is easier to stay out than get out.

You can also use the predefined tags (see Post Title, Post Content and Post Excerpt), which available for post title, post content and post excerpt templates. For example, if you want to include a link to source at the bottom of every generated post, you can do it by inserting the following line into the “Post footers” box:

<p><a href="%link%">Source</a></p>

Custom fields

Here you can assign XML tag values to custom fields of the syndicating post, using the following format:

tag->name

where:

tag – an XML tag name you want to select.
name – a name of WordPress post custom field to write the selected value to. This parameter is case sensitive.

The example below stores the values of <name> and <isbn> tags as the book_title and book_isbn custom fields:

name->book_title
isbn->book_isbn

You can also write a fixed arbitrary string value into a post custom field. To do so, you have to specify the sting value enclosed within double quotes and use it instead of the XML tag name. For example, this is how you can assign the “This is my product” string value to the product custom field:

"This is my product"->product

In order to assign XML tag values to the standard WordPress $post array fields. In this case the $post array field name must be enclosed within percent symbols. The rule below assigns the XML tag <name> value to $post['post_title']:

name->%post_title%

Expert Settings

CyberSEO Pro Syndicator Expert

Proxy mode

This option allows you to enable or disable proxies from the list, specified in General Settings. The proxies are usually used to parse those services that do not allow frequent connections from the same IP’s.

User agent

When you are trying to import a valid RSS feed, you may see the “Unable to acquire error” error message. This means that the owner of the feed has banned its import by automated scripts. In that case, it would most likely help you to substitute the user agent so that the CyberSEO Pro plugin can pretend to be a web browser or something else, such as Google Crawler.

Try the following values to avoid the mentioned above restriction:

  • Mozilla/5.0 (Windows NT 10.0; WOW64; rv:97.0)
  • Gecko/20100101 Firefox/97.0/3871
  • Googlebot/2.1 (+http://www.google.com/bot.html)
  • FeedValidator/1.3
  • PHP

HTTP referrer

Set URL for the referer spoofing. When set to “self”, the actual feed’s URL will be used.

HTTP headers

HTTP headers from a request follow the same basic structure of an HTTP header: a case-insensitive string followed by a colon (‘:’) and a value whose structure depends upon the header. One header per line.

XML section names (separate with commas)

Normally CyberSEO Pro automatically recognizes the internal structure of XML and JSON feeds, but sometimes the feed can be a very complicated one and it may contain a few different blocks of items, so you may want to define which exactly block you want to import. As you may know, the content of every RSS entry is enclosed within the <item> tags. E.g.:

<item>
    <title>Example entry</title>
    <description>Here is some text containing an interesting description.</description>
    <link>http://www.wikipedia.org/</link>
    <guid>unique string per item</guid>
    <pubDate>Mon, 06 Sep 2009 16:45:00 +0000</pubDate>
</item>

Thus any content automation plugin is able to properly recognize and parse the RSS feeds, since they have a very formalized structure. But if there is a need to parse an XML or JSON feed of an arbitrary format, you may face a serious problem because this task can’t be done by other content automation plugins. Fortunately this is not a problem for CyberSEO which will do it automatically. However you can always override its behavior using the “XML section tag names” field.

Let’s consider we need to parse some XML feed which entries are enclosed within the <product> tag and have the following structure:

<product>
    <id>PRODUCT-73182373</id>
    <name>DuroStar DS7200Q Remote Control Silent Diesel Generator</name>
    <specification>
        <img url="http://static1.ostkcdn.com/images/products/P12338714.jpg"><br />Perfect for the job site or the residential user who needs back-up power Features: Electric start on panel plus remote start. Advanced direct fuel injection system for low fuel consumption. Dependable, maintenance free alternator with automatic voltage regulator. Full power panel with keyed ignition, oil alert automatic shutdown for low oil pressure, volt meter, circuit breakers, low fuel indicator and power outlets. 12 hours Continuous Operating Capability. Fully protected 12V, 120V &amp; 220V outlets. Ground Fault Interrupter. DC 12V charging system.
    </specification>
    <price>$1,000</price>
</product>

The CyberSEO plugin will parse it automatically, but let’s pretend it was unable to recognize the <product> enclosing tag. In this case we need to go to CyberSEO->XML Syndicator and click the “Alter default settings” button. Now we have to locate the “XML section tag names” field, add there the new enclosure tag (in the case above it’s PRODUCT) and click “Update Default Settings”.

Done! Now the CyberSEO plugin will be able to recognize and parse any XML feed where all items are enclosed within the <product> tag.

However we still can’t pull this feed to the blog without some additional settings, because such important tags as <title> and <description> are missing there. Furthermore, there is an additional tag <price>, which we may also like to include into the post.

To solve this problem, we’ll use templates. First of all, we have to find the “Post title” box and alter its contents as follows:

%xml-tags[name]%

This will tell CyberSEO to use the XML value “name” as a WordPress post/page title.

Now let’s format the body of our post. Find the “Post content” box and put there the HTML template of the post you want to be generated. E.g.:

<p>%xml_tags[specification]%</p>
<p><strong>Price:</strong> %xml_tags[price]%</p>

Now we just need to click “Update Feed Settings” and voila – we are ready to automatically parse this XML feed as a source of new posts for our blog!

Disable JSON and XML item tag detection

The “Disable JSON section name detection” option allows you to disable the built-in sophisticated detection of JSON section names. And forces to plugin to use only the section names, specified in the “XML section names” box. How can it be useful? Let’s take a look at some ordinal JSON feed with the following structure:

{
    "feed": {
        "title": "Coming Soon",
        "id": "https://rss.itunes.apple.com/api/v1/us/apple-music/coming-soon/all/10/explicit.json",
        "author": {
            "name": "iTunes Store",
            "uri": "http://wwww.apple.com/us/itunes/"
        },
        "links": [{
                "self": "https://rss.itunes.apple.com/api/v1/us/apple-music/coming-soon/all/10/explicit.json"
            }, {
                "alternate": "https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewRoom?fcId=1396220241\u0026app=music"
            }],
        "copyright": "Copyright © 2018 Apple Inc. All rights reserved.",
        "country": "us",
        "icon": "http://itunes.apple.com/favicon.ico",
        "updated": "2020-10-25T02:16:04.000-07:00",
        "results": [{
                "artistName": "Miley Cyrus",
                "id": "1536966025",
                "releaseDate": "2020-11-27",
                "name": "Plastic Hearts",
                "kind": "album",
                "copyright": "℗ 2020 RCA Records, a division of Sony Music Entertainment",
                "artistId": "137057909",
                "contentAdvisoryRating": "Explicit",
                "artistUrl": "https://music.apple.com/us/artist/miley-cyrus/137057909?app=music",
                "artworkUrl100": "https://is3-ssl.mzstatic.com/image/thumb/Music124/v4/b9/c9/49/b9c949e6-d635-be2f-971d-81145effcf3d/886448810216.jpg/200x200bb.png",
                "genres": [{
                        "genreId": "14",
                        "name": "Pop",
                        "url": "https://itunes.apple.com/us/genre/id14"
                    }, {
                        "genreId": "34",
                        "name": "Music",
                        "url": "https://itunes.apple.com/us/genre/id34"
                    }],
                "url": "https://music.apple.com/us/album/plastic-hearts/1536966025?app=music"
            }, {
                "artistName": "The White Stripes",
                "id": "1534387685",
                "releaseDate": "2020-12-04",
                "name": "The White Stripes Greatest Hits",
                "kind": "album",
                "copyright": "℗ 2020 Jack White and Meg White",
                "artistId": "2456318",
                "artistUrl": "https://music.apple.com/us/artist/the-white-stripes/2456318?app=music",
                "artworkUrl100": "https://is5-ssl.mzstatic.com/image/thumb/Music114/v4/11/36/39/11363968-cd8d-96bc-e031-2b4a9a4743c4/886448804529.jpg/200x200bb.png",
                "genres": [{
                        "genreId": "21",
                        "name": "Rock",
                        "url": "https://itunes.apple.com/us/genre/id21"
                    }, {
                        "genreId": "34",
                        "name": "Music",
                        "url": "https://itunes.apple.com/us/genre/id34"
                    }],
                "url": "https://music.apple.com/us/album/the-white-stripes-greatest-hits/1534387685?app=music"
            }]
    }
}

If we let CyberSEO Pro to automatically detect section names, it will take the first one, which will be <links></links>. So we won’t be able to get the necessary information about the music albums. To solve this problem, we need to explicitly set the section we want to parse. This can be easily done as shown below.

  1. Add RESULTS_ITEM tag to the “XML section names”.
  2. Check (enable) the “Disable JSON section name detection” check box.

That’s all we need to do.

Preprocess source PHP code

This code will be executed every time when CyberSEO Pro reads the contents of a feed before it will be processed by the feed parser. Thу original content is stored in the $content variable. You can modify it in any way you want to fix formatting errors, add some parts to the content or delete something. This feature allows you to process just any type of content sources, including binary files or documents like PDF, Word documents, Photoshop documents and anything else. You just need to make sure that the resulting contents of the $content variable is formatted as a valid XML document.

If a site has no RSS feeds or even XML sitemaps, the source code preprocessing feature allows you to parse its HTML pages and turn them into a standard XML format, which can be easily imported just like a native RSS feed.

For example, the following code limits the syndicating RSS or Atom feeds by the first 5 items (posts) only:

$max_items = 5;
if (preg_match_all('/<(entry|item)>.*?<\/(entry|item)>/is', $content, $items)) {
    $content = '<rss><channel>' . implode('', array_slice($items[0], 0, $max_items)) . '</channel></rss>';
}

Yet another example of the preprocess source PHP code:

$xml = simplexml_load_string($content);
foreach ($xml->channel->item as $item) {
    if (isset($item->children('content', true)->encoded)) {
        $content_item = (string) $item->children('content', true)->encoded;
    } elseif (isset($item->description)) {
        $content_item = (string) $item->description;
    }
    $h2_tags = [];
    $dom = new DOMDocument();
    $dom->loadHTML($content_item);
    $h2s = $dom->getElementsByTagName('h2');
    foreach ($h2s as $h2) {
        $h2_tags[] = trim(strip_tags($h2->nodeValue));
    }
    $item->addChild('h2_tags', implode(',', $h2_tags));
}
$content = $xml->asXML();

This PHP snippet parses the RSS feed <content:encoded> and <description> sections for <h2> headers and adds them as comma-separated string enclosed in <h2_tags> section of the same feed item. So you can use these headers later in post templates or OpenAI assignments like as %xml_tags[h2_tags]%, which will be replaced by the actual comma-separated <h2> headers, extracted from the feed’s content.

Custom PHP code

This is the most powerful tool which is intended for advanced users who are familiar with PHP. Use it to execute your own PHP code for every aggregating post, which elements are represented as the $post array variable. You can alter this variable in order to apply your own changes to every aggregating post. The following items are defined in the $post array variable:

  • $post['post_title'] – post title;
  • $post['post_name'] – post slug name;
  • $post['post_type'] – post type;
  • $post['link'] – link;
  • $post['post_content'] – post content;
  • $post['post_excerpt'] – post excerpt;
  • $post['guid'] – GUID;
  • $post['post_author'] – post author;
  • $post['post_date'] – post date in Unix timestamp format;
  • $post['categories'] – array of post category names;
  • $post['media_content'] – array of media content URLs;
  • $post['media_thumbnail'] – an array of media thumbnail URLs;
  • $post['enclosure_url'] – enclosure URL;
  • $post['enclosure_type'] – type of enclosure URL. E.g.: audio/mpeg;
  • $post['custom_fields'] – array of custom fields;
  • $post['custom_fields_attr'] – array of XML tag attributes for corresponding custom fields;
  • $post['custom_taxonomies'] – array of custom taxonomies.
    E.g.: $post['custom_taxonomies']['taxonomy_name'] = 'term1, term2, term3, ...';
  • $post['comments'] – the array may contain post comments in WordPress format.
  • $post['custom_fields'] – array of post custom fields;
  • $post['custom_fields_attr'] – array of XML attributes associated with the post custom fields;
  • $post['tags_input'] – an empty array. You can use it to assign additional tags to syndicating posts via the $post[‘tags_input’] variable. E.g.:
    $post['tags_input'] = array('tag1', 'tag2', 'tag3', 'etc');

If you modify the $post array (say change the post tittle, insert some text into the post’s body, or create your own tags according to the post’s content), your changes will be applied to the post BEFORE it will be processed by CyberSEO and added to the blog. If you don’t want some particular post to be syndicated (say it contains some undesirable words etc), your PHP code must assign FALSE to the $post variable.

Examples

Make all post titles uppercase:

$post['post_title'] = ucwords(mb_strtolower($post['post_title']));

Add 12 hours to the post dates:

$post['post_date'] += 12 * 60 * 60;

Copy post content into post excerpt:

$post['post_excerpt'] = $post['post_content'];

Assign a random value between 1 and 100 (inclusive) to the custom field named “probability”:

$post['custom_fields']['probability'] = rand(1, 100);

Limit post content to 30 words and limit the excerpt with the first 25 words and add a link to source article below:

$max_content = 30;
$max_excerpt = 25;
$words = explode(" ", trim($post['post_content']));
if (count($words) > $max_content) {
    $post['post_content'] = implode(" ", array_slice($words, 0, $max_content)) . '<br /><a href="' . $post['link'] . '">Source...</a>';
} $words = explode(" ", trim($post['$max_excerpt']));
if (count($words) > $max_excerpt) {
    $post['post_excerpt'] = implode(" ", array_slice($words, 0, $max_excerpt)) . '<br /><a href="' . $post['link'] . '">Source.</a>';
}

Syndicate ONLY the posts that contain word1, word2 or word3:

if (!preg_match('/word1|word2|word3/is', $post['post_title'] . $post['post_content'] . $post['post_excerpt'])) {
    $post = false;
}

Do NOT syndicate the posts that contain word1, word2 or word3:

if (preg_match('/word1|word2|word3/is', $post['post_title'] . $post['post_content'] . $post['post_excerpt'])) {
    $post = false;
}

Add “The original article” link below every post:

$post['post_content'] .= '<p><a href="' . $post['link'] . '">The original article</a></p>';

AI article generation

The selected above AI engine will be applied to this feed. At the moment, the plugin supports two AI engines. These are OpenAI GPT-3 and Article Forge.

OpenAI GPT-3

CyberSEO Pro Syndicator OpenAI GPT

The OpenAI GPT-3 is a service that provides access to a suite of powerful language processing models developed by OpenAI. You may already be familiar with this technology if you have heard of or used the famous OpenAI’s ChatGPT service. These models can be used to perform a wide range of language-related tasks, such as translation, summarization, and question answering.

Using the CyberSEO Pro and OpenAI GPT-3 has several benefits. It allows you to supplement the syndicated posts with a relative unique content or simply generate the new articles based on given titles or descriptions according to your assignments. Additionally, the models available through the API are constantly being updated and improved, ensuring that you have access to the latest and most accurate language processing capabilities.

⚠ Important: OpenAI API is a 3rd-party service, so the content generation is being performed at remote servers. So it may take a rather long time to generate an article. Thus please you to not add more than one article at once. Also you should increase your maximum PHP execution time up to 400 seconds.

Article assignments

Set your free-form assignments for the OpenAI GPT, written in ordinary human language. CyberSEO Pro does not ask you to select an assignment language or the language you want to translate the generated text to. It does not make you choose the style of article, the keywords you want to be used from a set of drop-down lists with predefined options. Write them in a normal language, the same way you give an assignment to a human writer.

You can use the template shortcodes here: %post_title%, %post_excerpt%, %post_content%, %custom_fields[name]%, %xml_tags[name]% and %custom_fields[name]%.

Here is a few examples of the OpenAI GPT-3 assignment examples:

summarize this for a second-grade student:
%post_excerpt%
rewrite this article using keyword "economics":
%post_content%
write a sarcastic article on topic, translate it to German and break it into three paragraphs:
%post_excerpt%
write a sarcastic article on topic, using the "technology" keyword:
%post_title%
write a funny story on topic and convert all movie words into emoji:
%post_title%

And so on. As you can see, OpenAI GPT-3 understands a regular language in as same way as a human. So don’t limit yourself when writing your assignments!

You can set here an unlimited number of assignments, broken up with the <!--more--> separator string. Every such an assignment will generate a separate block of text, like this

Generate an introduction on the topic of:
%post_title%
<!--more-->
Turn the below into a H2 header:
%post_excerpt%
<!--more-->
Generate a sarcastic article on the subject of %post_title% using one of the following keywords "weather", "mood, "outdoors" that better fit the topic of the below::
%post_excerpt%

Note that Spintax is also supported. You can use it to make random assignments. For example, if you want to ask OpenAI GPT-3 to randomly use one of these keywords: “diet”, “fitness” and “healthy life” in the generated article, your assignment may look like this:

write an educational article on topic, using "{diet|fitness|healthy life}" keyword:
%post_title%

Article position

Set the generated article position. It can be inserted above the syndicated post or below it. Alternatively the AI-generated text can be used instead of the original articled.

Post title assignment

Give an assignment for the post title, or live it blank if you want to keep the original post title intact. You can set any assignment, which will be applied to the original post title, like:

translate the headline into Spanish using only capital letters

“modify this title: %post_title%”

“translate this title to German: “%xml_tags[some_xml_tag_name]%”

Model name

Currently CyberSEO Pro allows you to use the following OpenAI GPT-3 models:

Model Description Max request
Davinci (text-davinci-003) Most capable GPT-3 model. Can do any task the other models can do, often with higher quality, longer output and better instruction-following. Also supports inserting completions within text. 4,096 tokens
Curie (text-curie-001) Very capable, but faster and lower cost than Davinci. 2,048 tokens
Babbage (text-babbage-001) Capable of straightforward tasks, very fast, and lower cost. 2,048 tokens
Ada (text-ada-001) Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost. 2,048 tokens

While Davinci is generally the most capable, the other models can perform certain tasks extremely well with significant speed or cost advantages. For example, Curie can perform many of the same tasks as Davinci, but faster and for 1/10th the cost.

We recommend using Davinci while experimenting since it will yield the best results. Once you’ve got things working, we encourage trying the other models to see if you can get the same results with lower latency.

Max tokens

The maximum number of tokens to generate in the completion. The token count of each assignment plus Max tokens cannot exceed the model’s context length. Most models have a context length of 2048 tokens (except for the newest “davinchi” model, which supports 4096).

Temperature

What sampling temperature to use. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer.

Top p

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

Best of

Generates best_of completions server-side and returns the “best” (the one with the highest log probability per token).

Presence penalty

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.

Frequency penalty

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.

Article Forge

CyberSEO Pro Syndicator Article Forge

The plugin is integrated with a powerful Article Forge service, which automatically writes completely unique, on-topic, high-quality articles using advanced artificial intelligence (AI) and deep learning. You may use it to generate a whole content of your posts, or use it to enrich the posts syndicated from RSS feeds or other sources. E.g. you can add the AI generated articles above or below the main content of your posts. Just give it a keyword and Article Forge will generate an unique text for it.

The CyberSEO plugin allows you to define a desired keyword for every syndicated post, to set its position in the main post content. Also it allows you to use the AI generated article titles instead of the original ones. Note that post duplication check settings will be ignored if the “Replace the post content” option is selected.

⚠ Important: Article Forge is a 3rd-party service, so the content generation is being performed at remote servers. So it may take a rather long time to generate an article. Thus please you to not add more than one article at once. Also you should increase your maximum PHP execution time up to 400 seconds.

Article Forge API key

In order to use the service, make sure to set your personal Article Forge API key. If you don’t have one, you can obtain it here.

Keyword

A list of sub keywords separated by comma (e.g. subkeyword1,subkeyword2,subkeyword3). Article Forge has a five sub keyword limit per article. If you enter more than 5 sub keywords, Article Forge will randomly select 5. Article Forge will also ignore any sub keywords longer than 10 characters. Sub Keywords in should not contain URLs, parentheses, brackets, or too many single characters.

You can use template shortcodes here, e.g. the %post_title%.

Post tags as sub keys

When this option is enabled, the post tags (if available) will be used as subkeys for Article Forge.

Spintax view

Is useful to enable this option in order to generate the Spintax version of your article.

Article length

Select the desired article length from 50 to 750 words.

Article quality

Select the article quality: Standard, Adventurous, More adventurous, Conservative, More conservative.

Turing spinner

Enable or disable to Turing spinner.

Article position

Set the generated article position. It can be inserted above the syndicated post or below it. Alternatively the AI-generated text can be used instead of the original articled.

Image probability

The probability of adding an image into the article in percents.

Video probability

The probability of adding a video into the article in percents.

Article title

You can choose to leave the original post article or to use the Article Forge-generated one.

AI image generation

CyberSEO Pro Syndicator Article Forge

CyberSEO Pro is fully integrated with cutting-edge DALLE∙2 and Stable Diffusion AI image technologies,  that allow your to automatically generate astonishing pictures by a text description, written in an ordinary human language.

DALL∙E Image position

This drop-down list allows you to enable the DALL∙E 2 engine and specify position in the generated picture in article.

DALL∙E image assignment

Put your free-form assignment for For the DALL∙E 2 AI image generator. You can use a normal human language To describe what kind of image you want to get created. The assignment is limited by 1000 characters and your imagination only.

You can use template shortcodes here, e.g. the %post_title%.

DALL∙E image size

It is recommended to enable the Store images locally option. The size of the generated images. Must be one of 256x256, 512x512, or 1024x1024.

DALL∙E image class

Define the CSS class you want to use to display the generated image in order to set its size, align it or apply some visual effects to it.

Stable Diffusion image position

This drop-down list allows you to enable the Stable Diffusion engine and specify position in the generated picture in article.

Stable Diffusion image description

Put your free-form image description for For the Stable Diffusion image generator. You can use a normal human language To describe what kind of image you want to get created. Please read the basic tips on text prompt composing for Stable Diffusion AI image generator here.

You can use template shortcodes here, e.g. the %post_title%.

Stable Diffusion engine

Set the Stable Diffusion engine (model) to use for generation. Available engines:

  • stable-diffusion-v1
  • stable-diffusion-v1-5
  • stable-diffusion-512-v2-0
  • stable-diffusion-768-v2-0
  • stable-diffusion-512-v2-1
  • stable-diffusion-768-v2-1
  • stable-inpainting-v1-0
  • stable-inpainting-512-v2-0

Stable Diffusion clip guidance preset

CLIP guidance preset, use with ancestral sampler for best results. If preset is not selected “NONE”, Stable Diffusion will select one automatically. The following clip guidance presets are available:

  • FAST_BLUE
  • FAST_GREEN
  • SIMPLE
  • SLOW
  • SLOWER
  • SLOWEST

Stable Diffusion cfg scale

This value [0..35] defines how strictly the diffusion process adheres to the prompt text (higher values keep your image closer to your prompt). Default: 7.

Stable Diffusion steps

Number of diffusion steps to run: [10..150]. Default: 50. Affects the number of diffusion steps performed on the requested generation.

Stable Diffusion image size

Width [512..2048 ] and height [512..2048 ] of the image (note: height * width must be <= 1 Megapixel).

Stable Diffusion image class

Define the CSS class you want to use to display the generated image in order to set its size, align it or apply some visual effects to it.

Media handling

CyberSEO Pro Syndicator Media Handling

Store images locally

If enabled, all images from the syndicating feeds will be copied into the default uploads folder of your site. Make sure that your /wp-content/uploads folder is writable. This may speed up your site loading time and also lets you to syndicate posts containing hotlink-protected images.

⚠ Some sites don’t allow to hotlink their images. In this case the “Store images locally” option must be enabled so all these post images will be uploaded to your server (stored locally). Otherwise your posts will come up without images.

Store base64-encoded images

if this option and Store images locally are enabled, all the base64-encoded images will be saved as regular image files (.jpg, .png, .gif etc).

Media attachments

Some RSS feeds have so-called media attachments (enclosures) (usually images, videos and audios) enclosed within <media:content>, <media:thumbnail> and <enclosure> tags. The drop-down menu lets you decide on what to do with these types of attachments:

  • Do not insert attachments – the attachment will be ignored and won’t be inserted into the post.
  • Insert attachments at the top of the post – the attachment will be placed at the top of the post.
  • Insert attachments at the bottom of the post – the attachment will be placed at the bottom of the post.

In case if you decide to insert the attachments, they will contain in the <img> tag, so you will be able assign them some custom style.

Post thumbnails

When “Generate from the first post image” option selected, the CyberSEO plugin will generate post thumbnails (also known as featured images) using one of the following sources:

  1. First image found in the post content or excerpt.
  2. Last image found in the post content or excerpt
  3. Random image found in the post content or excerpt
  4. Media attachment thumbnail
  5. Contents of the “thumb” custom field

If no images found in the post content, the plugin searches for an embedded YouTube video to use its poster image as a post thumbnail.

Use FIFU

When enabled, the post thumbnail won’t be stored locally. It will be hotlinked and displayed by the FIFU “Featured Image from URL” plugin, which must be installed and activated.

⚠ Some sites don’t allow to hotlink their images. In this case FIFU must be disabled and all these post images must be uploaded to your server (stored locally). Otherwise your posts will come up without featured images.

Alternative thumbnail source

The alternative post thumbnail source for the case if the source image was not found. You can set here your default image URL or use the %random_image[]% shortcode. For example, use the %random_image[blue sky]% shortcode for a random image of the blue sky, or %random_image[%post_title%]% for an image, found by Google Images according to the current post title. This field will be ignored if the “Post thumbnails” option is set to “do not generate”.

Thumbnail source size

Set minimum and maximum thumbnail source image size in pixels. If the source image is smaller than a minumum limit or larger than a maximum one, the post thumbnail will not be generated. This helps you to sort out the posts with very small images when the post thumbnail is required. Use 0 if you don’t want to limit the thumbnail source image by size.

Post thumbnail is required

If the plugin will not be able to create post thumbnail as specified above (e.g. the source image is missing or broken), the post will not be added. So if you are going to enable the post thumbnail generation, please make sure that images are available in your content source.

This field will be ignored if the “Post thumbnails” option is set to “do not generate”.

Store videos locally

If enabled, all videos from the syndicating feeds will be copied into the default uploads folder of your site. Make sure that your /wp-content/uploads folder is writable. This my speed up the site loading time. Note that this operation may take some time, so the video content may not be immediately accessible. The following video file extensions are supported: mp4, m4v, mov, wmv, avi, mpg, ogv, 3gp, 3g2.

Use “video” custom field

When enabled, the contents of the video custom field will be embedded into the post. The custom filed may contain a direct link to a video file (.mp4, .webm ect) or an embed code (<iframe>). The following options are available:

  • Do not embed video – the attachment will be ignored and won’t be inserted into the post.
  • Embed video at the top of the post – the video player will be placed at the top of the post.
  • Embed video at the bottom of the post – the video player will be placed at the bottom of the post.

⚠ Do not enable this option if the “video” custom field is not defined in the “Custom fields” box.

Embed videos

Inserts tube video embeds into the post. Supported feed sources: YouTube, Vimeo, Flickr, IGN, DailyMotion. Supported RSS feed link examples:

https://www.youtube.com/feeds/videos.xml?user=vevo
https://vimeo.com/channels/wefoundthese/videos/rss
http://feeds.feedburner.com/ign/video-guides
https://www.dailymotion.com/rss/user/gqmagazine

⚠ This option is added for compatibility with CyberSEO Lite. In CyberSEO Pro it’s recommended to use presets instead. Also do not enable it if you don’t pull a feed from youtube.com, vimeo.com, ign.com or dailymotion.com.

Content filtering Settings

This section of the feed settings page allows you to filter syndicated content by keywords and phrases (separate them with commas), tags, publication date and text lenght.

CyberSEO Pro Syndicator Content filtering