next »

[ApoXX]

Below is yet another XML feed reader for MODx. Rather than being limited to specific formats, this snippet is capable of parsing and rendering nearly any XML format (RSS, RDF, Atom, etc). I call it FeedX (not to be confused with FedEx ). In addition to acting as a snippet, FeedX's class may be called by 3rd party snippets/plugins/modules as a simple method of retrieving cached and parsed XML data.

Updated: 2007/04/08

Snippet name: FeedX
Description: <strong>0.1.2</strong> General purpose XML feed reader

Code:

<?php
/*---------------------------------------------------------------------------
* FeedX Snippet - General purpose XML feed reader for MODxCMS
*----------------------------------------------------------------------------
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
* or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
* for more details.
*
* @author Brian Stanback (www.stanback.net)
* @copyright Brian Stanback 2007
* @created: 2007/02/27
* @updated: 2007/04/08
* @version 0.1.2
*
* Example usage: [!FeedX? &url=`http://...` &preset=`rss` &caheTime=`3600` &debug=`1`!]
*
*--------------------------------------------------------------------------*/
$feedx_version = '0.1.2';
$params = array();

if (isset($url)) $params['url'] = str_replace(array('|xq|','|xe|','|xa|'), array('?','=','&'), $url);
else return '';
// URL pointing to the feed to be displayed (stop snippet execution if not specified)
// The following URL characters need to be converted when passing them through the snippet
// ?  =>  |xq|
// =  =>  |xe|
// &  =>  |xa|

$params['preset'] = isset($preset) ? $preset : 'rss';
// Define a preset template to use...
// Presets are folders inside the /tpl/ directory, the snippet expects 
// the outer filename to be: $feedxPath/tpl/$tplPreset/outer.tpl
// (If outerChunk is defined, it will automatically take precedence over this value)

$params['outerChunk'] = isset($outerChunk) ? $outerChunk : '';
// Chunk to use for outer elements (default: use default rss template)
// Inner elements are called via chunk calls (in the template) formated like:
//  {{entityName->chunkName}}
// In addition, setting the maximum number of elements displayed can be done via:
//  {{entityName(max)->chunkName}}
// A start offset can also be added:
//  {{entityName(max,start)->chunkName}}
// XML values can be displayed within a chunk via placeholder calls:
// [+fieldName+]
// XML attributes can be displayed using:
// [+fieldName.attrName+]

$params['maxElements'] = isset($maxElements) ? $maxElements : '';
// Maximum number of times to display a repeating element
// This setting will be overridden by any limit values in the template
// Use the colon replacement value for element names which include colons
// Syntax: ELEMENT_NAME(x) where x is the maximum number of times to display
//         Multiple elements can be defined and must be separated by a colon
// Example: `CHANNEL(5):ITEM(10)`

$params['startElements'] = isset($startElements) ? $startElements : '';
// Define the start (offset) for the displaying of elements
// This setting will be overridden by any offset values in the template
// Syntax: ELEMENT_NAME(x) where x is the offset integer
//         Multiple elements can be defined and must be separated by a colon
// Example: `CHANNEL(1):ITEM(5)`

$params['filterElements'] = isset($filterElements) ? $filterElements : '';
// Specify a field to perform a match on, where the matching value is a Perl-style regular expression
// Matching elements can either be excluded using EQ (equals) or included NE (not equals) operators
// Syntax: ELEMENT_NAME(FIELDNAME NE /value/)
//         Multiple elements can be defined and must be separated by a colon
// Example: `ITEM(TITLE EQ /advertisement|sponsor|partner/)`

$params['sortElements'] = isset($sortElements) ? $sortElements : '';
// Specify a field to perform a natural sort on, if %rand% is specified the elements will be randomized
// ASC and DESC can be optionally be added to modify the order in which the elements are sorted
// Syntax: ELEMENT_NAME(FIELDNAME)
//         Multiple elements can be defined and must be separated by a colon
// Example: `ITEM(PUB_DATE DESC)`

$params['userPh'] = isset($userPh) ? html_entity_decode($userPh) : '';
// Set placeholder names and values which will be used globally
// Syntax: PLACEHOLDER_NAME->string value
//         Multiple elements can be defined and must be separated by a colon
//         Any string value with a colon should use the colon replacement
// Example: `JSID->test` ([+JSID+] contains the value test and can be used within all called templates)

$params['oddElements'] = isset($oddElements) ? html_entity_decode($oddElements) : '';
// If $outerChunk is defined the snippet will call a chunk, if not, 
// the snippet will attempt to use a file of the given name within the preset directory
// Syntax: ELEMENT_NAME->calledChunk
//         Multiple elements can be defiend and must be separated by a colon
// Example: `ITEM->oddChunk`

$params['evenElements'] = isset($evenElements) ? html_entity_decode($evenElements) : '';
// Syntax: ELEMENT_NAME->calledChunk
//         Multiple elements can be defined and must be separated by a colon
// Example: `ITEM->evenChunk`

$params['firstElement'] = isset($firstElement) ? html_entity_decode($firstElement) : '';
// Syntax: ELEMENT_NAME->calledChunk
//         Multiple elements can be defined and must be separated by a colon
// Example: `ITEM->firstChunk`

$params['lastElement'] = isset($lastElement) ? html_entity_decode($lastElement) : '';
// Syntax: ELEMENT_NAME->calledChunk
//         Multiple elements can be defined and must be separated by a colon
// Example: `ITEM->lastChunk`

$params['usePhx'] = isset($usePhx) ? intval($usePhx) : 1;
// Set to 1 to use the PHx parser (will fall back to standalone PHx mode if not installed globally)
// Set to 0 to disable PHx parsing

$params['convEntities'] = isset($convEntities) ? $convEntities : 1;
// Set to 1 to convert HTML entities to characters (default)
// Set to 0 to convert characters to HTML entities

$params['replaceColon'] = isset($replaceColon) ? $replaceColon : '|';
// Replace colons with the specified character(s) (defualt: |)
// (Resolves conflicts with PHx parsing)

$params['cacheType'] = isset($cacheType) ? intval($cacheType) : 0;
// Sets the level of caching
// 0 stores the the parsed data and re-renders on each page load
// 1 stores the parsed data as well as the finalized output for a small efficiency gain

$params['cacheTime'] = isset($cacheTime) ? intval($cacheTime) : 900;
// Time in seconds before re-retrieving a cached feed (default: 15 minutes)

$params['cachePath'] = isset($cachePath) ? $cachePath : $modx->config['base_path'] . 'assets/cache/feedx/';
// Path to the cache directory (default: modx_base/assets/cache/feedx/)

$params['timeout'] = isset($timeout) ? intval($timeout) : 20;
// Number of seconds to wait before giving up on feed retrieval

$params['feedxPath'] = isset($feedxPath) ? $feedxPath : $modx->config['base_path'] . 'assets/snippets/FeedX/';
// Path to the feedX snippet directory (default: modx_base/assets/snippets/FeedX/)

$params['language'] = isset($language) ? $language : 'english';
// Language for debug and error messages
// Language name must have a corresponding file in the feedxPath/lang/ folder

$params['debug'] = isset($debug) ? intval($debug) : 0;
// Set to 1 to turn on debug/template builder mode
// Helps users create custom chunks by showing all potential placeholder names in a hierarchal fashion
// Also shows information about loaded templates

/*--------------------------------------------------------------------------*/

$files = array(
'feedx' => $params['feedxPath'] . 'classes/feedx.class.inc.php',
'language' => $params['feedxPath'] . 'lang/' . $params['language'] . '.inc.php'
);
if ($params['usePhx'] && !class_exists('PHxParser')) $files['phx'] = $params['feedxPath'] . 'classes/phx.parser.class.inc.php';
if ($params['debug'])
{
$files['debug'] = $params['feedxPath'] . 'classes/debug.class.inc.php';
$files['debug_templates'] = $params['feedxPath'] . 'debug/debug.templates.php';
}

// Load classes
foreach ($files as $filetype => $filename) {
if (file_exists($filename))
{
if (strpos($filename, '.class.')) include_once($filename);
else include($filename);
}
else
{
if ($filetype == 'language')
{
$modx->logEvent(1, 3, 'FeedX: Language file does not exist: ' . $filename);
return 'FeedX: Language file does not exist: ' . $filename;
}
else
{
$modx->logEvent(1, 3, 'FeedX: ' . $_lang['file_not_found_error'] . ': ' . $filename);
return 'FeedX: ' . $_lang['file_not_found_error'] . ': ' . $filename;
}
}
}

if (!is_dir($params['cachePath']))  // Make sure cache dir exists
if (!mkdir($params['cachePath']))
{
$modx->logEvent(1, 3, 'FeedX: ' . $_lang['cache_creation_error'] . ': ' . $params['cachePath']);
return 'FeedX: ' . $_lang['cache_creation_error'] . ': ' . $params['cachePath'];
}

if (class_exists('FeedX'))
$feedx = new FeedX($params, $_lang);
else
{
$modx->logEvent(1, 3, 'FeedX: ' . $_lang['class_load_error'] . ': ' . $params['cachePath']);
return 'FeedX: ' . $_lang['class_load_error'] . ': ' . $params['cachePath'];
}

$output = $feedx->execute();

// Handle debugging
if ($params['debug'])
{
$debug = new FXDebug();
if (isset($_GET['dbg_dump']) && $_GET['dbg_hash'] == dechex(crc32($params['url'])))
{
$debug_html = $debug->render_popup($feedx, $feedx_version, $dbg_templates);
if ($_GET['dbg_dump'] == 'save')
$debug->saveDebugConsole($debug_html, $feedx_version);
else
exit($debug_html);
}
else
$output = $debug->render_link($feedx, $dbg_templates) . $output;
}

return $output;
?>

Download and extract the following file into your modx_base/assets/snippets/ directory: http://www.cyphergate.com/FeedX-0.1.2.zip

Demo: http://www.stanback.net/code/modx/feedx.html

This snippet is essentially an alternative to using something like XSLT stylesheet rendering. XSLT rendering is great for many situations but it lacks the ability to use PHx parsing, data limits/offsets, sorting, and filtering on individual feed elements.

Another interesting thing FeedX can do is convert a feed from XML to JSON or RSS to Atom, etc. using a series of custom templates/chunks. In addition, multiple feeds could be "merged" into a single feed by including multiple FeedX calls on a  MODx document with an XML template and custom http header.

Most of the pertinent information is documented within the snippet code above however I'll post more details on how to use this snippet shortly. Watch this post for updates.

[zi]

Great work ApoXX!

Thanks for sharing nice bits and bytes! ... and keep them coming . lol

best regards.

[Jesse R.]

I think Kyle is working on using your FeedX parser for GoogleEvents.  I was wondering what the snippet does in this case:

http://www.cmssandbox.com/MODx095/index.php?id=72

That feed has two [+LINK.HREF+] elements, and the second one overrides the first.  Maybe appending an ID to each would work?

[kylej]

Brian,
  I was working through a template for receiving a google calendar feed and have come across a problem.  When parsing a placeholder like : [+GD:WHERE.VALUESTRING+] it doesn't display anything unless phx is disabled.  I'm guessing that this is a limitation in phx with the : being in there causing the problem. 

Other than that looks like it works great though, one request though

Due to the complexity and necessary customization for a google calendar feed, I was wondering if you could add the functionality to just return the parsed xml.  this way I could call your snippet to get the xml and have it returned back in a nice php array.  That way I wouldn't have to duplicate the code into another snippet just to get/cache the xml.  Let me know what you think.

Kyle

[Jesse R.]

I think this would be a great addition also, and I would likely use it to update my WeatherX snippet.

Due to the complexity and necessary customization for a google calendar feed, I was wondering if you could add the functionality to just return the parsed xml.  this way I could call your snippet to get the xml and have it returned back in a nice php array.  That way I wouldn't have to duplicate the code into another snippet just to get/cache the xml.  Let me know what you think.

[ApoXX]

Thanks for testing. You're right, PHx is interpreting the colon as a modifier. The best solution at this point would be to simply replace colons with another character; how about a dash/minus or perhaps the pipe symbol?

I'd be glad to add a function for returning the cached/parsed data structure. You should also be able to access it like:
Edit...

$feedx = new FeedX($config);
$feedx->getData();
$mydata = $feedx->elements;

The cacheType must be set to 0 for this to work correctly; let me know what method for retrieving the data works best for you.

Also, what do you think about the current method of passing a configuration array to the class, would it be any easier for you if I also wrote a function for setting the configuration parameters?

[Jesse R.]

Thanks for testing. You're right, PHx is interpreting the colon as a modifier. The best solution at this point would be to simply replace colons with another character; how about a dash/minus or perhaps the pipe symbol?

Make it a parameter and it defaults to a pipe symbol would be my suggestion.

[kylej]

$feedx = new FeedX($config);
$feedx->execute();
$mydata = $feedx->elements;

this would be fine, however... if it could return the cached data if it exists that would be great also.  I have no problem with setting up a parameter array, so you don't need to change that.

Make it a parameter and it defaults to a pipe symbol would be my suggestion.

sounds like a good idea to me

[Jesse R.]

http://www.cmssandbox.com/MODx095/index.php?id=72

That feed has two [+LINK.HREF+] elements, and the second one overrides the first.  Maybe appending an ID to each would work?

Brian, what about this issue mentioned here?

[020200]

I don't know why, but I got an error when try parsing this feed: http://scnclr.de/?feed=rss2

The Feed above isn't working. The error is "Error retrieving feed."

I think the question mark in the URL makes the problem. I had a similar feed, but mod_rewrite to ./feed/ instead of ./?feed=rss2 solved the problem..

The strange thing is, that an URL-Questionmark replacement is at the beginning of the snippet and I think that PHx will also do some pre-rendering.

[kylej]

you have to replace the ? with |xq| and replace the = with |xe| and any & with |xa| in your snippet call

[ApoXX]

$feedx = new FeedX($config);
$feedx->execute();
$mydata = $feedx->elements;

this would be fine, however... if it could return the cached data if it exists that would be great also.  I have no problem with setting up a parameter array, so you don't need to change that.

Make it a parameter and it defaults to a pipe symbol would be my suggestion.

sounds like a good idea to me

It will automatically return the cached data if available and up to date, otherwise it will re-retrieve the data and populate the data array.

I'll go ahead and add a new configuration parameter for replacing the colon.

@jester444:

Thanks for bringing this up, I'll try to come up with a solution today. Mind sharing the original xml URL?

[Jesse R.]

Here it is...it is a test Google Calendar feed:

http://www.google.com/calendar/feeds/iblewyouup@gmail.com/public/full

[020200]

you have to replace the ? with |xq| and replace the = with |xe| and any & with |xa| in your snippet call

Ah, that worked! Thanks. Bw, is there a way to get that automated?

Another problem...

I want to build own templates as chunks from the modx-backend. I just copy'n'pased the rss .tpl files as single chuncks into the backend with the same names and tried to call it with &outerChunk='outer' . But this doesn't work. I just get a plain screen. Any help upon that?

[ApoXX]

Ah, that worked! Thanks. Bw, is there a way to get that automated?

Another problem...

I want to build own templates as chunks from the modx-backend. I just copy'n'pased the rss .tpl files as single chuncks into the backend with the same names and tried to call it with &outerChunk='outer' . But this doesn't work. I just get a plain screen. Any help upon that?

There currently isn't a way to automate the process of converting characters in the URL since the characters conflict with the snippet parser. It is possible to write a little web utility that would automatically convert the URL before you manually enter it into the snippet.

UPDATE: There was a slight problem in the snippet code when using chunks, I've updated it above. The downloaded files should not need changing.

[ApoXX]

I've updated the code/zip file in the first post of the thread to include colon replacement as well as adding an incremental integer when encountering duplicate items (within the same xml depth):

[+LINK.HREF+]
[+LINK.TITLE+]
[+LINK.1.TYPE+]
[+LINK.1.HREF+]

etc.

Let me know how it works.

@kylej:

To avoid unnecessary rendering when you are retrieving the data structure, I have created a separate function to only get the parsed xml data (it will also re-retrieve if the cache has expired). Please follow the below example:

Code:

$feedx = new FeedX($config);
$feedx->getData();
$yourarray = $feedx->elements;  // or reference $feedx->elements directly

If you need to change the URL or other configuration parameters used by FeedX, you can do it within a single instance rather than having to create separate instances every time:

Code:

$feedx->config['url'] = 'new url';  // Be sure to use ? & and = in this URL rather than replacing
$feedx->getData();  // Get new feed

By the way, my current template presets are pretty weak at the moment - if anyone wants to share their own creations I'd be glad to bundle them as the defaults.

[020200]

UPDATE: There was a slight problem in the snippet code when using chunks, I've updated it above. The downloaded files should not need changing.

Works!

[Jesse R.]

Seems to be working.  I have a demo running here:

http://www.cmssandbox.com/MODx095/index.php?id=57

@Brian:  To parse the start and end times into presentable formats, would I use PHx or would I have to access the array of elements like you described earlier?

@Kyle:  I am not working on integrating this into GoogleEvents, so don't worry about us working on the same thing.  I am just trying to wrap my head around the potential of this snippet.

[ApoXX]

You could probably use PHx, but you would have to use a custom (user-installed) modifier. Here is a little example which should be able to read in nearly any date and format it using PHP's strftime() syntax:

Snippet Name: phx:str2date

Code:

<?php
if (($time = strtotime($output)) !== false)
{
$dt = strftime($options, ($time + $modx->config["server_offset_time"]));
if ($modx->config['modx_charset'] == 'UTF-8')
    $dt = utf8_encode($dt);
return $dt;
}
else
return $output;
?>

Then inside your template/chunk:

Code:

<li><a href="[+LINK+]" title="Posted on [+PUBDATE:anydate=`%A`+]">[+TITLE+]</a></li>

I should also note that cacheType should be set to `0` during testing or your template changes won't show up until the cache expires. Also, any PHx modifiers that change dynamically (based on time of day, logged in user, etc) won't change until the cache expires if cacheType is set to `1`.

[pixelchutes]

Core integration here we come This is almost too powerful! Awesome!