User talk:Thingles

Archives: ... through January 2013.

AutoEdit in Foreground
Hi Jamie, hope this site is still thriving. Just thought I would drop you note since I'm currently re-designing my site, yellpedia.com, using the foreground skin. Hopefully I'll have it live shortly. Anyways I've noticed I seem to be having a problem with the autoedit function and was wondering if you've had any problems with it? Or any other problems that you would care to mention?All the best to you.Chris Tharp (talk) 06:33, 4 December 2013 (UTC)


 * Hello Chris! Sorry for the delay in reply. Foreground is doing great! I don't know of any issues with autoedit in Foreground. However, I am aware of many issues that have popped up over time with autoedit in general. By the way, there is now a foreground mailing list you may wish to join! 🐝 thingles (talk) 12:31, 10 February 2014 (UTC)


 * Hi Jamie. So I've found another issue with Foreground. Or at least it's an issue that I'm having: the preview function inside a semantic form doesn't seem to be working, This only happens inside the Foreground skin. Yaron suggested it maybe a javascript problem inside Foreground. Any suggestions you have on fixing this would be most helpful. All the best to you. (I joined the list)


 * Hi Jamie, I've found the problem with the preview function. It's caused by an extension I was using called Page Notice:http://www.mediawiki.org/wiki/Extension:PageNotice. May want to add that as a known problem for people using Foreground.

Re: Extension version grid
I agree, this extension grid is very pretty and can come of great use for the extensions which at least use versioning! All this "infrastructure" seems a bit wasted by the extension data not being pulled from the 4700 not validated wikis... I'd be so curious to see it ten times more full! --Nemo 09:06, 1 February 2013 (UTC)

Look forward to connecting
Hi Jamie, listened to a podcast, ready for more. Would very much like to connect for a few minutes via voice. Best, MarkDilley (talk) (of WikiIndex)


 * Hey there Mark! Would be fun to connect. Not sure where you are, timezones and all. You can find me on Skype under . You can also find me on Freenode in #mediawiki as   pretty frequently. I'll shoot you my email as well via the wiki. Thingles (talk) 20:52, 3 February 2013 (UTC)

MediaWiki versions
Heiya Jamie, I just managed to remove the MediaWiki from the version picked up by Bumble Bee. However this is only a quarter of the bet and MediaWiki does not use "-" to label special versions, alphas, betas or rcs. I guess some Regex is needed to get things going. What we need is 1.21 which collects 1.21wmf, 1.21alpha and 1.21rc (all of these may be regarded as non-production and fit into one count) and 1.21.0, etc. Hmm, this is something for Master T, I guess :). Can you figure out a Template:Get simple version number for this? Cheers --&#91;&#91;kgh&#93;&#93; (talk) 18:41, 7 February 2013 (UTC)


 * This is exactly what I'm working on in User:Thingles/Scratch4. I'm referring to those labels as "flags". As you can see, it mostly works. My biggest issue is that for some reason "2.2" is parsing into the first and fourth variable, rather than the 1st and 2nd. I plan on this regex method replacing Template:Get simple version number. I'll put this further up my list. The "flag" versions are bothersome and skew things a lot. Thingles (talk) 19:38, 7 February 2013 (UTC)


 * Hey, that's cool. I knew it was somewhere in the pipeline. :) These "flags" are indeed a pain in the neck. :( I was asking because we have that many different versions in the meantime that the statistics page gets pretty crowded. There we should just display the branches like 1.20.x etc. and show the details on individual (sub-)pages. I believe the statistics page will gain from this a lot (less is more). The really interested people may proceed to the exact breakdowns. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 19:47, 7 February 2013 (UTC)

Skins on Wikia
Heiya Jamie, the API implementation for skins seems to cause problems for skins on Wikia or Wikia does some strange things. Bumble Bee constantly rearranges the order of the skins according to what it picked up. This creates a new revision every day though the selection of skins stays the same. So far I have not seen this for skin data retrievals on other wikis. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 12:09, 13 February 2013 (UTC) PS I have just seen this on another non-Wikia wiki there it happens. Probably the API then. I wonder why this does not happen in every case. --&#91;&#91;kgh&#93;&#93; (talk) 16:40, 14 February 2013 (UTC)


 * Thanks for highlighting this. I think I'll have to manually sort skins before I put them into the wiki. I'm not going to deal with this for a while. It's causing unneeded edits but otherwise no harm. Thingles (talk) 02:58, 26 February 2013 (UTC)


 * It is just that all these revisions enlarge the database with not additional benefit at all. Indeed, not a top important issue. --&#91;&#91;kgh&#93;&#93; (talk) 01:01, 27 February 2013 (UTC)


 * This seems to happen as soon as code updates for existing skins or extensions get moved onto the server. Since Wikia is probably doing it every day ... The sorting will be the best way to solve this. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 22:48, 9 March 2013 (UTC)

$wgNamespacesToBeSearchedDefault
Heiya Jamie, I think it will be useful to set the "Extension:", "Farm:" and "Skin:" namespace to true. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 21:22, 13 February 2013 (UTC)


 * Brilliant! Done! Thingles (talk) 16:22, 14 February 2013 (UTC)


 * Perhaps not brillant, but helpful. :) Thank you. --&#91;&#91;kgh&#93;&#93; (talk) 16:38, 14 February 2013 (UTC)

Re: Updates!
Great! I hope to see all the Pavlo's sites updated with data by the bot soon: they're such a tiny part of the wiki world, we need to move on. ;-) --Nemo 09:18, 25 February 2013 (UTC)

Concepts within concepts
Heiya Jamie, you might like bug 44467. :) --&#91;&#91;kgh&#93;&#93; (talk) 00:09, 26 February 2013 (UTC)


 * :-) Was bummed when that didn't work when I tried it. I added a vote on that bug! Thanks! Thingles (talk) 02:53, 26 February 2013 (UTC)

Collection frequency for wikis < 1.11.0
I guess a daily run (1440) is more than sufficient for these wikis since the collectable general information for them is stable. Probably a week (10080) would do the job too. What do you think? --&#91;&#91;kgh&#93;&#93; (talk) 00:36, 26 February 2013 (UTC) PS What about adding version specific categories to the website pages, e.g. Category:1.9.0 for all wikis of version 1.9.0. This would make text replacing much easier. Doing an aggregated Category:1.9.x in a second step will be nice too. Probably to much voodoo to add the aggregated category right away instead of version specific ones. --&#91;&#91;kgh&#93;&#93; (talk) 00:36, 26 February 2013 (UTC)


 * Something feels wrong about adding categories for semantic data. :-) But, I might be being dumb. It would be nice for certain queries for sure, and definitely Replace Text. However, User:Audit Bee is now doing most of what we would do version dependent replace text on. You feel strongly about this? Thingles (talk) 19:44, 2 March 2013 (UTC)


 * I actually described a dirty voodoo workaround since ReplaceText only allows to narrow down results by namespace or category. However I strongly feel positive about reducing the number of visits to wikis with versions lower than MW 1.11.0. If there would be a Bee (e.g. Dirty work Bee ;) which is checking for the wikis in question and changes the update frequency this would definitely be the better solution. --&#91;&#91;kgh&#93;&#93; (talk) 22:45, 2 March 2013 (UTC)

Vote -> boom
I clicked the vote button for Wikidata and stuff went boom, see the history there :) --Jeroen De Dauw (talk) 00:45, 2 March 2013 (UTC)


 * Thanks for letting me know Jeroen! This is nasty behavior in Semantic Forms when the  flag is being used, seemingly in conjunction with a , and probably only when intersecting with an   call. Phew! You are not in the operator group (no offense. ;-P) but the autoedit seems to still allow you to do the edit (note your vote change did occur), but the other fields that were set to restricted reverted back to their defaults. Bummer. For now, I've removed the restricted parameters. They've caused a handful of challenges and I don't think they are really necessary (at least for now). Thanks for checking in on WikiApiary! I just enabled another bot to audit and activate sites automatically so the data set here is going to get way more robust in the coming days! (PS, I've filed bug 45644 on this issue.) Thingles (talk) 19:42, 2 March 2013 (UTC)

System change
Heiya Jamie,

I was very bold an bought myself an new computer. This resulted into a new OS and the transition turned out to be a very painful process. More than expected. Basically this is the reason why I was not very active here lately.

Cheers --&#91;&#91;kgh&#93;&#93; (talk) 18:58, 6 March 2013 (UTC)


 * Very cool! What did you get yourself? What new OS? BTW, you picked a good time to be less active. Audit Bee is slamming the site and load on my server is 4x higher than normal. Audit Bee should be done by Thursday or so, and then things will return to normal. The very first audits with page creations have a higher load. See WikiApiary page graph. :-) Thingles (talk) 20:11, 6 March 2013 (UTC)


 * Hmm, very cool...? Well, this is a matter of taste. I switched from NT 5.1 to NT 6.2, but since I was using the Win 95 GUI for 18 years now, even on NT 5.1, this is kinda big change for me. Still I will probably install a Squeeze soon since I bought a 2 TB harddrive and now have plenty of space. As an old and quaint habit I still compress most of my files.
 * Yeah, Audit and Bumble Bee are very hard working bees right now and it is indeed getting a bit frantic on your server. :) I am surprised that there are that few problems apart from the JSON decoding. --&#91;&#91;kgh&#93;&#93; (talk) 22:39, 6 March 2013 (UTC)

Website form broken?
Just tried adding a wiki using the Website form. On submission, I just got a blank page, and no actual save operation occured. --Jeroen De Dauw (talk) 01:41, 7 March 2013 (UTC)


 * Damn! Sorry about that Jeroen. I pulled SyntaxHighlight GeSHi from master today and it looks like there was a change that made it dependent on 1.21.x. I see my error log had

PHP message: PHP Notice: Use of undefined constant CONTENT_MODEL_JAVASCRIPT - assumed 'CONTENT_MODEL_JAVASCRIPT' in /srv/www/mediawiki/public_html/w/extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.php on line 76


 * …over and over. It looks like it was also causing edits via the API to fail which means auditing wasn't working either. Thanks for letting me know. I've switched that extension to it's  branch now which hopefully will keep things happier. Thingles (talk) 02:18, 7 March 2013 (UTC)


 * Yeah, looks like this commit broke me. Thingles (talk) 02:35, 7 March 2013 (UTC)


 * Ugh... looks like my bots triggered this error 39,627 times today. :-( Thingles (talk) 02:20, 7 March 2013 (UTC)


 * Odd, it looks like API autoedit's worked when this problem was present. Thingles (talk) 02:27, 7 March 2013 (UTC)

The bees really have done their job well
It is great to see that these two workaholic bees have done such a good job. :) Which lot is next? ;) I have posted here about the great changes to the display you did today. There are also a couple of suggestions. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 22:52, 9 March 2013 (UTC)


 * The first audit of every site is done! The next step for automation is to get the error fields being used properly. Right now none of the bots set it, so soon I will have then incrementing those fields and then figure out when Audit Bee should mark them as defunct. Need to add a function to apiary.py to do this. The other item that is high on my priority list is to start creating the daily and weekly summary data in the database and start allowing those to be charted. I've got an idea for a very cool set of graphs using the daily and weekly data that just won't work with the raw data as it is collected now. I also need to add an audit bee process that prunes things like the Bot log which will now grow forever. That's pretty easy though. The graphs need a ton of help, but they need it all in Javascript which isn't a terribly strong point for me. Might see if I can get someone to help out on that. Oh, and the data PHP scripts too. :-) Thingles (talk) 13:19, 10 March 2013 (UTC)


 * True, automated error handling should really be the next step now that all the wikis are on board and active. This alone will at least reduce the error log and reduce the work operators will have to do manually. It is so sad that I do not know JavaScript since I would have loved to help here. My JavaScript is basically as good as your Swahili. :( --&#91;&#91;kgh&#93;&#93; (talk) 22:58, 10 March 2013 (UTC)


 * First step in this direction finished tonight. See this diff and here is the commit with the functionality added. Will watch this for a while. Next step will be to clear the error flag (right now it won't reset). Then would be having Audit Bee start deactivating sites that are over a certain error count. Thingles (talk) 03:16, 11 March 2013 (UTC)


 * Almost forgot a cool thing I'm adding to Bumble Bee too. I figured out that I could sort of cache the response for things like skins by storing a SHA-256 hash of the result the remote wiki gives me. So, I'm going to add this to Bumble Bee so that he will stop barraging WikiApiary with  edits for things like extensions and general info. Instead he'll do a SHA-256 on the API result from the target wiki and will only attempt a save to WikiApiary when the hash doesn't match.  Nifty! I'm also changing the refresh rate for those data sets. Right now they all check every 24 hours. I'm going to have general update every 3 days, extensions daily and skins every week. I'm also going to add a random offset to it to make sure that the segments don't general hot spots. Thingles (talk) 14:45, 10 March 2013 (UTC)


 * Once again a goodie. :) New revisions for no cause are really not a good thing to have. The hash approach is as nice as it is nifty I guess. :) Support from me for the change in update frequency though you could even change the skin updates to a fortnight. This would elegantly solve the < 1.11 "issue". --&#91;&#91;kgh&#93;&#93; (talk) 22:58, 10 March 2013 (UTC)

Swap jobqueue with edits
ATM the "job queue" graph spans the whole page while edits only half of it. Why not swap them? Seems like edits is more interesting to most people. in fact, I guess a lot of people do not even know what the job queue is all about :) --Jeroen De Dauw (talk) 16:48, 13 March 2013 (UTC)


 * Yeah, definitely. You are seeing a bit of work in progress there. The plan is to just have those three larger charts, and have a control to allow the user to switch what data they wish to view in those. Included in that is the link to popup those charts into a separate window (instant dashboard!). You are right... I doubt many care or even know about their job queues. Thingles (talk) 02:34, 14 March 2013 (UTC)


 * Just swapped edits in. See WikiApiary. :-) Thingles (talk) 02:42, 14 March 2013 (UTC)


 * I care! While it shouldn't be too visible, it's very useful. If you made a graph also for each farm, including all Wikimedia projects, I'd be so happy! You'd fix . --Nemo 16:33, 14 March 2013 (UTC)


 * That was fixed properly at last. ;) --Nemo 21:13, 3 May 2013 (UTC)

Whitelist
Could you please whitelist traditio-ru.org? Is a good site, honest. Alex Mashin (talk) 06:46, 16 March 2013 (UTC)


 * Sorry about that. I was using the lists from meta for Extension:SpamBlacklist and it's definitely a bit aggressive. Traditio-ru.org is activated and will start collecting very soon. Thingles (talk) 13:21, 16 March 2013 (UTC)

Alex Mashin (talk) 20:04, 16 March 2013 (UTC)
 * Thank you.

Bookmarks
Are the bookmarks at http://links.thingelstad.com/wiki/Main_Page collected manually or automatically? (impressive job, by the way). Alex Mashin (talk) 20:20, 16 March 2013 (UTC)


 * Thanks Alex! They are entered manually (although semi-automated using a bookmarklet). I previously used Pinboard.in, and before that Del.icio.us. I had thought that Semantic MediaWiki could support a nice link wiki so I made one and then wrote an importer to pull all my previous bookmarks over. It even has a bot that updates URL redirects and checks the health of links. I have a bot that will replicate that wiki to others (see my friend User:Garrickvanburen's one at http://links.garrickvanburen.com/). Thingles (talk) 22:42, 16 March 2013 (UTC)

Installed version/latest version in Extensions section
Hi Jamie,

when I look at Special:Version of wikis I often wonder what is the latest version of the installed extensions. I think it would be nice to have that on WikiApiary, meaning on the Extension section/tab of each wiki's page not only display the installed version of each extension but also what would be the latest version of each extension. --Eric (talk) 12:59, 18 March 2013 (UTC)


 * Thanks for the suggestion Eric. This is definitely in my sites. It may first happen as an email for site administrators. If you have a Web site on WikiApiary that you would want that for edit it with form and add a notification for yourself. Also, you can click through to the Extension pages themselves and it will often have the version info there. But it would be much better to put it right in the Web site page. I'm just wary that it will invoke a query for every extension that the site has. Probably fine... 🐝 thingles (talk) 02:11, 19 March 2013 (UTC)

Questions/suggestions
Is it practical to get Bumble Bee to automatically upload and add logos when they're missing? At least if they're in a usual place. Adding logos takes significantly more time than the rest of the add wiki process, even when they're easily collected up like I had with the NIWA wikis.

Also, about "Allow wikis to use WikiApiary to do weekly backups. (Likely a paid service.)", if space becomes not much of an issue would you consider doing an annual XML text backup for free for non-huge wikis, but not retrievable without a one off charge (used to fund more backups)? Many wiki admins don't take care to backup, and as a result wikis are lost. And have a list of wikis which have gone down that you've got a backup off available. That'd be awesome, practically ensuring that no wiki is ever entirely lost.

And did you see my suggestion on MediaWiki_Extension? Is that likely to be practical long term?--Ete (talk) 17:10, 20 March 2013 (UTC)

Not semantic?
This wiki is semantic: OpenEI --Jeroen De Dauw (talk) 19:14, 21 March 2013 (UTC)


 * Was removed by User:Kghbln along with collect skin data, I imagine something about that wiki makes the SMW API inaccessible? I'll try turning it on, and note errors in the edit reason if it's still not working. (hope it's cool to reply to things like this on your talk page thingles)--Ete (talk) 19:38, 21 March 2013 (UTC)


 * Totally cool. I would tend to put a thread like this on Talk:OpenEI, but doesn't matter a ton. 🐝 thingles (talk) 20:20, 21 March 2013 (UTC)


 * When OpenEI was first added and Extensions pulled it was Feb. 15, 2013 and it was using Semantic MediaWiki 1.5.1.1 at that time (rev 25258). SMW 1.6 or greater is needed for  and hence the sentiment of User:Kghbln's comment "that's sad". On or right before March 11 it looks like they upgraded to Semantic MediaWiki 1.8 (rev 59051). So, now we can collect Semantic data! Yeah! Now, furthering the story. User:Audit Bee visited this site on March 8 to do an audit. On March 8, they were still running SMW 1.5.1.1 so everything was left as is. If nobody would have noticed this, User:Audit Bee would have enabled semantic data collection on it's next audit around June 9, 2013. 🐝 thingles (talk) 20:20, 21 March 2013 (UTC)


 * That's exactly what happened. Great observation. Beginning of February they also had big trouble with availability. This was the reason why I acted that late on them. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 22:10, 21 March 2013 (UTC) Currently I am pretty busy. This is why you do not see me around that often these days. Will be back though.

Wikia is dreaming about this :)
See here - or they have traffic from outer space. :D --&#91;&#91;kgh&#93;&#93; (talk) 19:24, 27 March 2013 (UTC)


 * Ha! Indeed! 🐝 thingles (talk) 22:26, 28 March 2013 (UTC)

API help
First, great site and use of SMW! Love being able to see stats, something lacking out of the box on MW installs without really searching for them.

Second, I noticed you have a stat on edits per week. How are you grabbing that? We are trying to come up with a way API edits per month, but I'm failing to grasp the what and how without hitting the revision table directly with a direct query. Appreciate a pointer in the right direction. Thanks --- Hutchy68 15:31, 5 April 2013 (UTC)


 * Hey there! You can see the calculation of that index in Template:Website indexes. In short, I have the active user count for now and from 7 days prior ($$activeusers\prime$$), as well as the edit count from now and 7 days prior ($$edits\prime$$). And then I do:
 * $$(edits - edit\prime) \div ( (activeusers + activeusers\prime) \div 2 )$$
 * I can do this because I store the user and edit values over time from the API, so I can use the differential. I'm not sure how you would do this if you were calculating it by querying the MediaWiki database directly. Hope that helps. All of the site indexes are ones I made up, with the exception of Depth. So I would be happy to hear of any suggestions for improvement. 🐝 thingles (talk) 20:29, 9 April 2013 (UTC)


 * Thanks for the info, I'll take a look at the template. For now we are using a complete API to render a plotted wiki activity chart on our domain. I think we have a work around for 'time period' plotting by a direct db query. We are investigating SQL answer now, but we are trying to get the next version release out so this is taking a back-seat for the next week or so. I did run across an old statistical extension on a MW 1.11 install called MonthlyStats. I think I'll throw it up on Github and work on it as it is GPL. It uses Google charting, very detailed, but could be better.  Thanks again! Hutchy68 (talk) 14:38, 19 April 2013 (UTC)


 * Very cool. For what it's worth, you could access the stored data in WikiApiary for your website. I'd be happy to show you how to pull the time series data out. Also, right now I'm only storing the raw data but "real soon now" I'm going to be adding daily and weekly information so I can show edits per day and edits per week on a graph with a 0 axis. The current edits that just grows "up and to the right" isn't terribly useful. You really want to see if there is a slump or jump in edit activity. (I'm also considering an email alert on that type of thing.) You are doing this per user though, which is something I don't have any plans to integrate with WikiApiary. Edits by user would be a massive dataset. I could maybe (!) see adding it for sites that have some sort of premium subscription. 🐝 thingles (talk) 14:57, 19 April 2013 (UTC)

Hi thanks for creating an interesting site
Hi, just stopped by to say thanks for creating an interesting wiki. Love the concept of the site; anything that gives more publicity to all the wikis out there should be celebrated.

Sadly the data about my site, yellpedia.com, is wildly off due to a flaw in the design of how the magic word works (see my note on the talk page for yellpedia.com). Not sure anything can be done about this, or if it should be done, since my issue is most likely unique. Regardless I wish you well, and hope this site thrives. Chris Tharp (talk) 04:00, 9 April 2013 (UTC)


 * Thanks for the complement Chris! I responded at Talk:Yellpedia. 🐝 thingles (talk) 20:23, 9 April 2013 (UTC)

Vision?
Hi Jamie! You're doing very good job - the job WMF have to do, in fact. What vision so you have for Wikiapiary? I'm particularly interested in extension and skin management, do you plan to add anything for them? Yury Katkov (talk) 18:53, 14 April 2013 (UTC)


 * First off — thank you for the donation Yury! That is generous of you and totally awesome! I would be happy to enable more frequent data collection for a few wikis that you care about. By default User:Bumble Bee only checks websites every 4 hours but I'd be happy to enable 15 minute collection for some if you would like. Let me know.
 * Regarding vision I probably have more of that than I have time. If you dig back on talk pages you'll see that initially I wasn't sure about bulk adding a ton of sites but that turned out to be awesome. I would like to add as many wikis as possible to WikiApiary but it's not my top priority right now. If others wanted to write spiders to add them I would happily help with the code and run them. I'm focusing more on improving the site itself, continuing to improve the bots, and hopefully soon getting User:Notify Bee working so I can start sending emails to people when they have old software or interesting things happen on their wikis. I've also considered adding some more streams of data. The stats that I pull along with extensions and skins are good. I think I could do some really interesting anti-spam work if I pulled in the list of new users on each wiki.
 * In terms of vision, I've been following closely the conversation on the mailing lists about better extension management. I think WikiApiary is in a great position to help provide data for that. I don't believe anyone else knows what version of what extensions are being used on what versions of MediaWiki. WikiApiary does. And I think the recent addition of related extensions provides some more utility. I would be very interested in WikiApiary being the API/data repository for the questions that a more robust extension manager would need to answer. I've been hesitant to jump into this quickly though, just being cautious and don't want to be presumptive.
 * In other areas of vision I would ultimately love to also develop a WikiApiary extension that people could install to tie even more information from WikiApiary into their sites. Not sure what all it would do, but I'm very interested in detected spam registrations and remotely nuking the accounts from WikiApiary.
 * I should be clear that I do see a commercial opportunity here too. I fully plan to offer backup services and other help for admins and I see there being a free, basic and premium offering. The main variations for those services will be: how long is data kept for the wiki, how deep is the backup that WikiApiary keeps and how frequently will your site be checked.
 * I guess I jumped into backups without mentioning that I am starting to work on backing up ALL wikis (smaller than say 10,000 pages for now). I plan to only backup the current version of pages (no history) and only the text. Thing of this as a time capsule archive. This would be a free version of backup, but if your site were to die and you need it there would be a one-time fee to get it.
 * Anyway, that is a start. I'm all ears on vision and where WikiApiary could go though. I'd love to know what you think about the items above and what other thoughts you have for WikiApiary! 🐝 thingles (talk) 02:15, 17 April 2013 (UTC)


 * Wow! The backup services would indeed be very nice and potentially profitable. And the notifications about the old versions of the software where the critical bug or major security risks have been found is very useful as well. Have a look at that, those guys seems to do backups/archives as well. Yury Katkov (talk) 16:54, 17 April 2013 (UTC)


 * Aww, too bad you're not planning to work on a crawler directly, I have to find someone else starting work on that! I'm glad you're willing to help, though. (I came here wondering about that.)
 * As for the archive/backups, I've already backed up all the wikis we currently have on wikiapiary (those coming from Pavlo's list), they're at https://archive.org/details/wikiteam . What we need is 1) a more complete list of wikis, 2) faster software, for which you can see WikiTeam/Dumpgenerator rewrite. Archiving only last revision is dangerous because often (very often!) it's a vandalised/spammed one, some wikis are completely erased by spam but the history has everything. --Nemo 21:09, 3 May 2013 (UTC)

Ranking/voting? Contests?
What do you think about ranking the websites, extensions and skins? I see that you have now some kind of ranking system based on Semantic Forms, but it's very primitive. We in WikiVote are now releasing two extensions for voting for opensource and I would be eager to install one of them on wikiapiary. The voting widgets can look like this: http://test2.wikivote.ru/images/3/30/Voting3.png and you can very flexibly set up what kind of voting do you want. Yury Katkov (talk) 16:58, 17 April 2013 (UTC)


 * Yury, I would love to get better crowdsourced ratings for extensions, skins and possibly even the wikis themselves. The primitive featured site vote count that is in WikiApiary now is only intended to be a way to weight a future content box on the front page that would rotate through featured websites (similar to featured articles on Wikipedia). As a way of providing a robust database of extensions knowing usage information and versions is key, but having some way for users to rate the extensions is a big missing piece right now. I'd love to see what you have going on. The existing extensions I've looked at for rating pages are a bit sketchy. Ideally I also want something that is able to be semantically queried. If only a mythical Semantic Ratings existed! :-) 🐝 thingles (talk) 15:22, 19 April 2013 (UTC)


 * Cool, so it's my turn to open the code of our extensions and show you the demo. And yes, we know how to semantize stuff :) Yury Katkov (talk) 21:15, 20 April 2013 (UTC)

Sudden drop in Uncyclopedia stats
&rarr; Moved conversation to Talk:Uncyclopedia.

U_of_I_Wiki_2
Hi, I forgot to check if the page already existed, would you mind deleting it? Thanks, Legoktm (talk) 11:19, 21 April 2013 (UTC)


 * No worries. Done. By the way, do you think there should be a template? 🐝 thingles (talk) 11:21, 21 April 2013 (UTC)
 * I'm not sure how often you get deletion requests, its just that's the template I'm used to existing on all the wiki's I typically edit. Legoktm (talk) 09:13, 24 April 2013 (UTC)

$smwgAdminRefreshStore
Heiya Jamie, I would set this parameter of SMW to false. Not that you are in danger of me doing this her, but I believe that it should not be possible to do it for regular wiki admins. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 20:00, 25 April 2013 (UTC)


 * Great catch. Corrected. Thanks. (I run a refresh via cron for all my wikis.) 🐝 thingles (talk) 20:22, 25 April 2013 (UTC)


 * Yeah, cron is the way to do it. What still irritates me is that regular wiki admins can initialise or upgrade the store. I guess there should be a setting for this, too. Hmm..., will have to check if there is at least a user right for this. --&#91;&#91;kgh&#93;&#93; (talk) 20:36, 25 April 2013 (UTC)

Make graphs more readable
Hi Thingles,

I suggest you could add colors to the y axis of the graphs. That would make the graphs more readable. In Pages and Articles graph for example: add green color to Articles and blue color to Pages.

I saw there is a colored mouse-over, but I think y axis colors would still help... Stefahn (talk) 09:58, 26 April 2013 (UTC)

Browse by extension version
I was just checking out Extension:Semantic MediaWiki and wondering which wikis where currently running version 1.8.0.4. Would be cool if there was a way to easily drill that down :) --Jeroen De Dauw (talk) 22:09, 16 May 2013 (UTC)


 * I'm planning on adding some convenience links to make that simple. 🐝 thingles (talk) 05:23, 26 May 2013 (UTC)

Property duplication
Heiya Jamie, I have seen the issue reported with this bug 48707 on this wiki, too. Perhaps you would like to vote or comment on it. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 12:59, 22 May 2013 (UTC)


 * Thanks. I commented and voted. 🐝 thingles (talk) 19:44, 22 May 2013 (UTC)

DFRV - Regionalgruppe Berlin
I am a bit puzzled. Even though Bumble Bee correctly picks up the MediaWiki version, this information does not get set on the website's page via property "Has MediaWiki version". I guess a second pair of eyes ... ;) Cheers --&#91;&#91;kgh&#93;&#93; (talk) 10:28, 24 May 2013 (UTC)


 * Yeah, Bumble Bee just updated subpage "General" and now the property is properly set. However, it should have been here before. Never mind - paradise regained. :) --&#91;&#91;kgh&#93;&#93; (talk) 11:35, 24 May 2013 (UTC)


 * I did see this while it was in error and it made no sense to me at all. Bumble Bee resaved the page, which caused SMW and MW to redo their work and then it was there. Very suspicious, but not worth investigating if we can't reproduce. 🐝 thingles (talk) 05:22, 26 May 2013 (UTC)


 * Indeed since we already had a couple of resaves before. Just keeping fingers crossed. --&#91;&#91;kgh&#93;&#93; (talk) 06:54, 26 May 2013 (UTC)

File upload
Totally broken.

[618cf099] 2013-05-26 08:05:26: Fatal exception of type MWException Alexander Mashintalk 08:07, 26 May 2013 (UTC)


 * Sorry about that. I enabled  and the issue was obvious right away. After upgrading to 1.21 it seems Extension:PDF Handler breaks all uploads (not just PDF!). I created bug 48834 and have for now disabled the extension entirely. Thank you for letting me know! I also uploaded the logo for Encyclopedia Dramatica, partly as my test. 🐝 thingles (talk) 11:40, 26 May 2013 (UTC)

Fixed properties
Heiya, I have commented here. Probably you would like to have a peep at it too. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 16:05, 26 May 2013 (UTC)


 * Definitely and have done now. 🐝 thingles (talk) 16:16, 26 May 2013 (UTC)

Graphs gone
The two graphs here are gone: Farm:Wikimedia --Jeroen De Dauw (talk) 14:08, 29 May 2013 (UTC)


 * Thanks Jeroen. I got rid of the bubble charts (I was never a fan of them) and replaced them with a more complex bar chart of the farm and it's children. It still needs work and is work in progress, but I think it's better. Would be curious what you think. Also, there is something very odd going on with jqplotseries. I opened bug 48944 on it. I'm seeing nasty Javascript exceptions in Safari and Chrome, but not Firefox. Seems to have happened after 1.21 upgrade. James HK sent me an email on it related to Statistics not having data labels, but I'm also seeing this on smw:Demo:Jqplotseries/Survey as well. 🐝 thingles (talk) 16:08, 29 May 2013 (UTC)

Bumble Bee
I saw this page in recent changes, and I know that Extension:GoogleDocs4MW already exists. Simant (talk) 17:21, 7 June 2013 (UTC)


 * Yeah, saw that come through too. The odd thing is that the 4 wikis that supposedly use it don't seem to have entries for it. Debugging needed. Not sure if it's Bumble Bee or a big in SMW/SF. Thanks! 🐝 thingles (talk) 19:52, 7 June 2013 (UTC)


 * Okay, I think this is a bug in Semantic MediaWiki and/or Semantic Forms recreating this page. If you dig into the 4 sites that are supposedly using this extension they don't actually show this one in use. I'm guessing it got recreated as an echo of some data that is still in the Semantic MediaWiki store but is no longer relevant. 🐝 thingles (talk) 13:36, 19 June 2013 (UTC)

a major security risk
Hello! This wiki uses extension Regex Fun that have a major security risk


 * Thank you for the note. I've disabled the extension. I don't even use it on WikiApiary but I do on other wikis in my farm. Can you elaborate on the security risk associated with this extension? 🐝 thingles (talk) 13:37, 26 June 2013 (UTC)

Cross promotion
Hi Jamie,

We exchanged messages before, but just to remind you: I run a wiki called Yellpedia, which has the entire United States Yellow Pages in it. Anyways I'm doing a lot of changes on my site to make it more user friendly over the next few days and one of things I'm doing is adding a form for websites to add their info to my site. So I was thinking maybe I'll add a special form just for wikis, which leads to my thought that maybe we could cross promote. I could add a message to my wiki form to make sure they go here to fill out their info and maybe you could do the same. Regardless of your decision I wished you continued success in this project. All the bestChris Tharp (talk) 16:54, 7 July 2013 (UTC)

300%?
See Extension:Diff. Version 0.7alpha has 300%. What? ;p --Jeroen De Dauw (talk) 22:58, 14 July 2013 (UTC)

Extension grouping
It'd be nice if the extensions used for a given website could be displayed in a more organized fashion (rather then the unordered list currently in use). For this extension extension types as on Special:Version could be used, though perhaps it is nicer to use the types/tags they have assigned here on Wikiapiary --Jeroen De Dauw (talk) 22:58, 14 July 2013 (UTC)

Extension renaming
I might well have brought this up already...

Extension:Validator_(ParamProcessor) and Extension:Validator are the same extension. Soon I will change the name to "ParamProcessor (Validator)" and at some point (many months from now), it'll change to "ParamProcessor". Would be nice if this did not end up as 4 distinct extensions here :) --Jeroen De Dauw (talk) 23:10, 14 July 2013 (UTC)

Active sites
Right now there are a bit more then 8000 active sites registered. Any idea what fraction of public MW installs this is? What is the kind of growth the number of registered wikis is seeing? Is there a kind of auto-discover thing going on via some search engine or crawler? Answers to these questions are interesting when gauging the adoption of MediaWiki in general. --Jeroen De Dauw (talk) 17:45, 15 July 2013 (UTC)
 * Most wikis tracked are part of farms we've been able to get lists of wikis, added by hand, for or from the Pavlo import project, which was a google crawler run in 2008. I suggested rerunning it, and the source code is available, but thingles prioritized improving WikiApiary over crawling for more wikis. If someone were to rerun that bot it would give a much better picture of how many MW installs there are, and the rest could be added (maybe gradually, depending on the scale of the import needed).--ete (talk) 15:04, 22 July 2013 (UTC)
 * If there's any interest still, I may be interested in re-running the script. --Tux (talk) 10:49, 24 July 2013 (UTC)

Better ShoutWiki Support
Hi Thingles,

I was wondering if I can work with you to get better support for ShoutWiki here on WikiApiary. We have an API module to get a list of all our public wikis, however it's not exactly the cheapest thing to run regularly, so I don't want to advertise it really publicly. I'm happy to discuss this over IRC, I'm Lcawte(|Away) on Freenode and I'm almost always in #mediawiki & #shoutwiki.

--Lewis Cawte (talk) 14:10, 22 July 2013 (UTC)
 * Hey, I'm not sure if there's been any update on this since there are plenty of shoutwiki wikis missing. Would it be alright if I ran a bot to create pages for the wikis that are currently missing? I need to play with the SMW API a bit, but I don't think it'll be that difficult. Thanks, Legoktm (talk) 05:59, 3 February 2014 (UTC)
 * Hi lego, I think this is not necessary: I've just uploaded dumps for all public shoutwikis to archive.org, so User:WikiTeam will add them at the next run. --Nemo 07:23, 3 February 2014 (UTC)
 * Yeah, the next time the bot runs it will log the api.php "misses" and then I run those through an import cycle so that will get them all in. It will take the next week before the WikiTeam archive information is attached (since the wikis aren't there on the first run). I would like to get a proper farmer bot in place for ShoutWiki, but I would recommend waiting on that for now. 🐝 thingles (talk) 13:09, 3 February 2014 (UTC)
 * Yay, that sounds awesome. Thanks Nemo! Legoktm (talk) 20:43, 4 February 2014 (UTC)

Thank you for your work
WikiApiary is impressive and very useful. Keep up with the good work! Sadly my pet project is nanosmall but still I'm happy to contribute with a bit of data.--QuimGil (talk) 19:34, 21 August 2013 (UTC)

Comprehensive list of interwiki prefixes
Hi Thingles, thanks for the informative message about the differences between WikiApiary and WikiIndex. What do you think of implementing a centralized Canonical interwiki prefixes list via WikiApiary? I would have to do some research to figure out how to make this interact with SMW. But basically, what I have in mind is that someone would change the interwiki table by editing, e.g., the RationalWiki article on WikiApiary and adding a template (e.g. ) or SMW property, e.g. has interwiki prefix::rationalwiki.

After the saving of the revision, the interwiki table would be checked to see if this prefix already has been added by another page (e.g. a rival project claiming the same prefix); if so, the first-comer would get to keep the prefix, but the conflict would be noted in an iw_conflicts database page. If the first-comer's interwiki prefix were to be changed/removed, then the second-comer would automatically get the prefix.

Other wikis could poll to get the list of prefixes and their associated URLs. There would be a heck of a lot of interwiki prefixes (over 9,300) so to avoid the need to poll the API 19 times (to get 500 prefixes at a time) to get the latest list, capability for incremental updates would have to be added. (1) This could be done through the logging table or (2) a new field could be added to the interwiki table noting when a prefix was added. (In the case of option 2, however, it would also be necessary to have a table noting what prefixes have been removed and when.)

The goal of all this is to enable wikis across the wikisphere to keep their interwiki tables up to date with the centralized one maintained by WikiApiary that can be modified by the users by editing the various wiki description pages. There are other ways to implement this too, but this strikes me as having the advantage that it only requires one change to add all the necessary data for a wiki (description and prefix) rather than two (one to add the description and another to add the prefix, as would be the case if one had to go to Special:Interwiki).

I think we should try to agree on some specifications, and then coding can be begin. I already have created mw:Extension:InterwikiMap, which I use on most of my wikis. Another summary of this proposal, which called for using templates rather than SMW, is here. Thanks. 14:43, November 23, 2013‎ User:Leucosticte


 * This seems like a great idea to me, and it’s very easy for me to implement. I just need to create an additional property (as you suggest above) for each wiki called Has interwiki prefix and allow that to be modified on the form. I wonder if there might be a way for a bot to even figure out what it might be for a given wiki. The template could also check to see if any wikis have a prefix that are in conflict and place those in a category for resolution. The last step that would be fun is to have another bot build a file that anyone can easily import into their wikis to fill their interwiki tables. The first part is easy enough. Only real question I have is should a wiki be able to have multiple prefixes defined? What do you think? 🐝 thingles (talk) 01:43, 24 November 2013 (UTC)


 * Okay, I just went ahead and added this. Should be done now. See Property:Has interwiki prefix, change to Template:Website and change to Form:Website. I set the prefix for WikiApiary. Seem like the right approach? 🐝 thingles (talk) 01:55, 24 November 2013 (UTC)


 * Also shared this on WikiApiary-l. 🐝 thingles (talk) 01:58, 24 November 2013 (UTC)
 * Cool, I just signed up for that listserv. I don't know too much about SMW (my experiments with SMW ending in my running up against its limitations fairly quickly, so that I had to abandon the project I was working on that would have depended on SMW). I can see situations arising in which, say, RationalWiki and ReptileWiki both might want rw:. So, you have to come up with a procedure for resolving conflicts. Maybe deny that short prefix to both of them and require that they have prefixes that completely spell out their names.


 * I don't see why a wiki would need a bunch of different prefixes, unless one of them is an abbreviation of the other, or the wiki changed its name (e.g. Disinfopedia becoming Sourcewatch). I don't know what to do about situations in which, say, there are four different RationalWikiWikiWikis. That's a real head-scratcher; I guess pick the most important of the wikis to get the best prefix, and the others have to figure something else out, but I'm not sure what. That kind of situation shouldn't arise too often, though. Leucosticte (talk) 02:40, 24 November 2013 (UTC)

ViewFiles
I'm feeling kind of nosy about WikiApiary's configuration; do you think you could install Extension:ViewFiles? When the configuration database becomes a core feature, LocalSettings.php will mostly become obsolete, but in the meantime I think it's good to give users a peek at the files, the better to kibitz about the settings. Leucosticte (talk) 02:40, 24 November 2013 (UTC)

Bugzilla or another issue tracker
What do you think of the idea of setting up an issue tracker for WikiApiary, given all the suggestions people might have for new features? I'm hoping that eventually Extension:IssueTracker becomes ready for deployment but it needs a lot of work before it can succeed to the throne. So much development to do, so little time! In other news, I added you as a cc on bug 57059 because I think that a wiki such as this could be like an anti-spam Interpol for the wikisphere. And I'm not talking about the band! Leucosticte (talk) 02:40, 24 November 2013 (UTC)

Monitored by
Hi Thingles, would you allow my wiki to use your campaign image File:Monitored_by_WikiApiary.png directly instead of hotlinking to WikiApiary? I.e. I would upload the file to our wiki and use it as picture in the footer that links here. I'm a big fan of reducing hotlinking/embedding content from other domains to a bare minimum in order to reduce possible security risks and/or conflicts with browser security extensions like NoScript and RequestPolicy. Of course I'd only use the image in the footer/reference to WikiApiary and give all credit and copyright/license information pointing to you. Have a nice Xmas and enjoy the holidays, --Theaitetos (talk) 22:29, 18 December 2013 (UTC)


 * Yeah. That is totally fine. Thanks for asking. Merry Christmas to you as well! 🐝 thingles (talk) 10:54, 19 December 2013 (UTC)


 * Thanks, [//www.wiki-aventurica.de/wiki/Datei:Monitored_by_WikiApiary.png all done]. Another question: Our wiki is 15min down every night for maintenance, from 23:55 to 0:15 (UTC+1). Is it possible to tell the little bees to only query information outside of this time frame? --Theaitetos (talk) 17:15, 20 December 2013 (UTC)


 * That is an interesting situation. No, currently there is no way to tell Bumble Bee to not collect during a certain time period. I could see people wanting this if they felt Bumble Bee was taxing their server too much during the day as well. But, it’s just a few API calls so that seems unlikely. If you want to create an issue on Github for this maybe it would get added in the future. 🐝 thingles (talk) 02:29, 22 December 2013 (UTC)

A thousand more non-farm mediawikis
Hello! As you may remember, I'm trying to extract URLs to mediawiki wikis from search engine(s). So far I have about 1200 api.php URLs, about 1000 of which were not in the old list by Pavlo which you previously imported: can you add them? Pavlo's list is almost 5 years old now, this is smaller but has what search engine(s) give today. It would make wikiapiary more representative of today's situation and also help me understand what sort of wikis I'm finding. --Nemo 10:31, 4 January 2014 (UTC)


 * This is awesome Nemo! They are importing right now in Category:January 2014 Import! 🐝 thingles (talk) 15:02, 4 January 2014 (UTC)


 * This is completed. The output of the import is archived at [[File:January 2014 Import Log.txt.zip]]. Category:January 2014 Import shows 662 new wikis added. Awesome!


 * Hah! Less than a thousand, but could have been worse. :) I'm impressed at how quick you were, wikiapiary is getting very robust. Wonderful. --Nemo 20:16, 4 January 2014 (UTC)


 * It is! And when the new bots that use a distributed message queue get in place get ready because we should be able to then load Farm:Wikia. :-) 🐝 thingles (talk) 14:10, 5 January 2014 (UTC)


 * So wonderful that I was motivated to play with some silly regexes and I got about 1400 new URLs added to . Do you feel like running a new import? :) --Nemo 10:52, 5 January 2014 (UTC)


 * Very cool! I found the comm command and pulled the new entries into their own file. It is processing right now. See Special:Contributions/Audit Bee! 🐝 thingles (talk) 14:10, 5 January 2014 (UTC)


 * Great work you guys are doing; thanks. I guess there are quite a lot of extensions that aren't documented over at MediaWiki.org and whose code isn't shared with the rest of the world. I guess that just shows the truth of the Two Types of Programmers concept. Leucosticte (talk) 13:17, 5 January 2014 (UTC)


 * This batch has now been imported! Thanks again Nemo! This batch added 1,178 websites. There are currently 982 waiting for their initial audit which should finish in about 20 hours or so. The output of the import can be seen at [[File:January 2014 Import Log (part 2).txt.zip]]. Nice work! 🐝 thingles (talk) 16:51, 5 January 2014 (UTC)

I just noticed a mistake with : the archived wiki was shut down and just redirects to the one the page is about. As a curiosity, the size changed because I added a review to the item, which added to metadata. --Nemo 15:32, 18 January 2014 (UTC)


 * And now I extracted about 4k more wiki URLs! Half of those are index.php URLs, which may have a corresponding api.php (just without RSD i.e. pre-1.17) or not. I don't remember if you're also able to pick stats from Special:Statistics. --Nemo 21:00, 27 January 2014 (UTC)


 * WOW. Cool cannot wait to see them roll in. It must be really nice for the developers and the others intensely involved with the development of MediaWiki to see how many installations are out there. Wonder how many of all existing wikis are already aboard. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 21:16, 27 January 2014 (UTC)


 * Oh, and if it matters I can easily check which of those index.php URLs actually exist in api.php too. --Nemo 07:38, 28 January 2014 (UTC)

Namespaces
Hi, I notice that siteinfo allows namespace data to be gathered. I think it would be interesting to get a list of the different custom namespaces various wikis have. For example, some have an Essay: or Miscellany: namespace. Who knows what other kind of namespaces are out there? One might be able to find intriguing content just by searching by namespace and finding wikis, or searching by wiki and finding namespaces. Leucosticte (talk) 02:14, 26 January 2014 (UTC)


 * Hey there! Namespaces were one of the things that I planned to collect when first working on WikiApiary. You can see some discussion on it at User talk:Kghbln, see point 4. I think it could come back into the plan as part of the revamped bots. It’s definitely easy to collect. 🐝 thingles (talk) 03:08, 26 January 2014 (UTC)

WikiApiary quite broken on Internet Explorer 8 (on Windows XP)
Hi Jamie, it appears that WikiApiary's layout is rather broken on Internet Explorer 8 &mdash; the last version of IE available for Windows XP. As such, it's reasonable to assume that IE8 will be "the new IE6", despite that the official extended support for Windows XP ends in April.

On a page like Extension:MobileApp, I get the following notices in IE's console:

LOG: ']' is needed Extension:MobileApp, line 293 character 36 LOG: Exception thrown by ext.echo.base TypeError: 'config.get(...).eventlogging' is null or is not an object TypeError: 'config.get(...).eventlogging' is null or is not an object

The first one, which is related to the Piwik analytics code, is also the easiest to fix. In this particular case, line 293 was  &mdash; there's some incorrect escaping going on in there.

Additionally, there's a separate "Object does not support this property or method" JS error on the Main Page, caused by  (line 2, character 1499). I initially thought that the issue was that the code in question is trying to use the HTML5 data attributes, but based on the chart at CanIUse.com as well as some quick googling, it seems that IE8 should be able to live with the  attributes...

On a page about a wiki, such as OpenAnesthesia.org, there are these two JS errors in addition to the standard Piwik one:

'CanvasRenderingContext2D' is not defined https://wikiapiary.com/apiary/lib/dygraph-combined.js, line 2 character 13300 The supplied value must be an object https://wikiapiary.com/apiary/graph/graph.js, line 361 character 5

If you need any screenshots or someone to test code on IE8, let me know &mdash; whilist I no longer use IE8 as my primary browser (thankfully!) nor Windows XP as my primary OS, I have access to plenty of XP computers.

There's also browsershots.org, which is a good resource for cross-platform testing (and the software's FOSS, too!), but alas, there are limits on how many screenshots you can request from one IP address and sometimes the screenshots either outright fail or are useless (a screenshot where the majority of the web page is blocked by a "Upgrade Flash Player" window isn't very useful).

Finally, thank you and please keep up the awesome work! WikiApiary is a great resource which provides additional data way beyond what the extension and skin pages at MediaWiki.org are capable of &mdash; it's always nice to see how many wikis are running the code I've written and which versions are the most popular ones. ;-) --Jack Phoenix (talk) 08:52, 20 February 2014 (UTC)

Correctly use the Skin: namespace for skins instead of the Extension: namespace
I noticed that there are two separate pages for Aurora, ShoutWiki's default skin; one at Skin:Aurora and another at Extension:Aurora. It's true that skins can be made to show up at Special:Version and I definitely register the skins I've written so that they show up there for consistency's sake, but on WikiApiary, they should be located in the Skin: namespace only. This is especially true when there's both a skin and an extension with the same name &mdash; somewhat of an edge case, but it can happen (cf. Extension:Nimbus vs. Skin:Nimbus). --Jack Phoenix (talk) 09:53, 20 February 2014 (UTC)