WikiApiary talk


From WikiApiary, monitoring the MediaWiki universe

← Previous month Next month →

WikiApiary:Bot log

The revamp of the bots that I'm working on made some good progress. Notably, the bots now log to ApiaryDB and there is now a page at WikiApiary:Bot log that uses Extension:External Data to show recent information on the wiki. I suspect this will be a huge win for operators. Thingles (talk) 04:34, 2 February 2013 (UTC)

It's worth emphasizing that this is only for the new version of the bot. The current one running does not log here, so until I get the new one doing the day-to-day work this will just show my debug runs. Thingles (talk) 04:42, 2 February 2013 (UTC)


Today I put in an entire new namespace for Skins, along with Form:Skin, Template:Skin and other miscellaneous stuff to track skins. Check out the Category:Skin to see. Some unique things to take note of. Skins can have related extensions associated with them (see Skin:Vector). Also, skins can indicate that they were derived from another skin (see Skin:Vectorata). Thingles (talk) 04:36, 2 February 2013 (UTC)

Transcluding to get refresh faster

I made a change today to Template:Website to purposefully transclude the bot generated subpages for a Web site even when they don't exist. I'm hiding them in a span so the user can't see, but MediaWiki sees it and properly notes that the page is using a red link template. The new version of Bumble Bee will pull these pages on demand, instead of just once a day. Doing this transclusion will hopefully result in a faster refresh when the pages are updated on demand from the bot. Thingles (talk) 04:41, 2 February 2013 (UTC)

Example of intentional redlinks.

Add SMW Wiki of the Months?

I added the recently announced SMW Wiki of the Month using a specific tag. Would be cool to tag all the historical wiki of the months as well. Thingles (talk) 12:39, 3 February 2013 (UTC)

This can be done. However, I think there is only one other of them here. The rest still has to be added. --[[kgh]] (talk) 13:10, 3 February 2013 (UTC)

Bumble Bee cleaning up Extension authors

When I added the code to automatically populate Property:Has author I noticed right away that my simple method of just placing the string returned by the API in there had some challenges. Most people use commas to separate names so that worked well, but it is fairly common to put wikitext in the author field so that authors names are clickable. This caused all sorts of problems when showing automatically generated author groupings. So, I put some regular expression filters in to clean these up before they are added to WikiApiary. The block looks like this:

# Authors can have a lot of junk in them, wikitext and such.
# We'll try to clean that up.
temp_author = x['author']
# Wikilinks with names
# "[[Foobar | Foo Bar]]"
temp_author = re.sub(r'\[\[.*\|(.*)\]\]', r'\1', temp_author)
# Simple Wikilinks
temp_author = re.sub(r'\[\[(.*)\]\]', r'\1', temp_author)
# Hyperlinks as wikiext
# "[ Jeroen De Dauw]"
temp_author = re.sub(r'\[\S+\s+([^\]]+)\]', r'\1', temp_author)
# Misc text
temp_author = re.sub(r'\sand\s', r', ', temp_author)
temp_author = re.sub(r'\.\.\.', r'', temp_author)
temp_author = re.sub(r' ', r' ', temp_author)
# Lastly, there could be HTML encoded stuff in these
temp_author = h.unescape(temp_author)

template_block += "|Extension author=%s\n" % (temp_author)

This helped a lot and you can already see improvements. Very importantly, this keeps square braces from getting through which caused problems in the templates themselves. If you think of other filters like this that would help let me know and I can easily add them. This list of hundreds helps see what they look like now (note that a full SMW Refresh that happens every night should clean up stragglers). The main downside of this are authors that use handles as opposed to names, but that is where overriding the automatic generation with manual authors is a better tact. Thingles (talk) 01:41, 5 February 2013 (UTC)

100 semantic sites tracked

As of today 100+ websites are tracked by WikiAPIary that use Semantic MediaWiki --[[kgh]] (talk) 23:38, 11 February 2013 (UTC)

Very cool! And it's 138 now! :-) Thingles (talk) 16:29, 14 February 2013 (UTC)

1000 sites validated

As of today 1000+ websites have been validated on WikiAPIary, basically meaning that they have been checked for being still online. Additionally most them were tagged and got a short description. --[[kgh]] (talk) 21:45, 14 February 2013 (UTC)

1000 sites actively tracked

As of today 1000+ websites are being actively tracked by WikiAPIary. --[[kgh]] (talk) 00:20, 16 February 2013 (UTC)

200 most popular extensions tidied up

As of today the 200 most popular extensions have been tidied up, basically meaning that their home was verified or added, that the collected data were streamlined and that short descriptions were provided. --[[kgh]] (talk) 12:36, 16 February 2013 (UTC)

Extensions of more than 1000 sites tracked

As of today WikiAPIary tracks the extensions of more than 1000 wikis. Thus 1330 different extensions were collected so far with the most popular being used on 673 wikis. --[[kgh]] (talk) 01:14, 28 February 2013 (UTC)

So cool! Great milestone! Thingles (talk) 15:02, 28 February 2013 (UTC)