User talk

Thingles/Up to January 2013

From WikiApiary, monitoring the MediaWiki universe

< User talk:Thingles

Logos on this wiki

Heiya Jamie, why not allowing them to be embedded rather than uploading all of them here? Cheers --[[kgh]] (talk) 20:19, December 28, 2012 (CST)

Good question. My thought was that the data for a wiki may end up outlasting the wiki itself and the logo image being local would insure that it is available as long as this wiki is. I could support both with a two different fields, or even just detect if the page exists and if it doesn't assume there is a URL in the field. Do you feel strongly remote URL's should be supported? Thingles (talk) 20:22, December 28, 2012 (CST)
That's true and a valid idea. I was more thinking from a copyright point of view here. Since this wiki is CC BY-SA and most logos are copyrighted there may arise issues. I believe this can be circumvented by always tagging them as copyrighted an adding a note about logos to the copyright information page of this wiki. Another way to avoid such issues would be to allow remote URL's. Cheers --[[kgh]] (talk) 06:58, December 29, 2012 (CST)
Do you think that the note on the bottom of this blog post for Wikivoyage could be an example of how to deal with this? Thingles (talk) 17:48, 15 January 2013 (UTC)
Yes, however the note should be a general one so it fits to every logo here. Something like perhaps: "This image is one of the official logos or designs used by the organisation or individual running the respective wiki listed on WikiApiary. The use of the logo and trademark is most likely subject to the trademark policy and visual identity guidelines of the organization or individual it owns, and may require permission." It must be imho clear in any case that the logo does not fall under the general license set for WikiApiary (CC BY-SA). Thus it can in my opinion be ruled out that somebody says that WikiApiary released it under CC BY-SA. Another note should probably indicate that WikiApiary will remove the logo immediately if it is asked by the copyright owner to do so. Besides, isn't there some fair use policy over at your place which would help here, too. I cannot tell if this will be bullet-proof, crazy people are all over the place, but is it leaps better than the current situation here. Cheers --[[kgh]] (talk) 18:09, 15 January 2013 (UTC)

To Do for Thingles  on 1/1/2013
Decide on tagging wiki logos as copyrighted material.

I case we run out of ideas which wiki to add

Heiya Jamie, I do not know if you are aware of this stats portal. There you can also see which information they collect. However, I already like this wiki better. :) Cheers --[[kgh]] (talk) 07:07, December 29, 2012 (CST)

I found that when doing a little searching (I linked it on my bookmark wiki). It seems like there are a couple of efforts (and are a couple of abandoned projects as well) to track usage across multiple wikis. I envision WikiApiary overlapping a little with those, but not much. For example, I put the initial set of wikis into apiary so I would have a test set, and I also think it would be crazy to not have the main Wikimedia wikis present. But I don't want to load lists of thousands of wikis. I found the lists for Wikia and such, and I could bulk load them, but I'm not sure that is where I want this to go.
Those projects are just numbers and dashboards. I haven't seen anything that looks at versions of MediaWiki and what extensions are installed. I intend to add email features to WikiApiary so you will get an email with weekly and monthly reports on activity. I also want to send emails when versions of extensions or MediaWiki that a site is using are out-of-date. I'm considering adding a feature to do a weekly backup of a wiki (this would require a small fee).
Indexing activity on wikis is cool, and there will be some of that. But I'm putting this together in part to answer my own needs (and I think the needs of other wiki admins). What happened on my wiki this week? How active is my wiki? Are my extensions current? Is my wiki backed up? Thingles (talk) 07:33, December 29, 2012 (CST)
PS - Thanks for the early praise too! :-)
I believe you have done your homework. :) s23 tried to retrieve usage numbers for extensions but this was soon broken. Just recently they removed the link to the usage stats from the extension's template on I think to have usage stats for extensions is a great information to have both for developers and users. This wiki is indeed the only one of which I know that caters for this, apart from the smw community wiki which has a much narrower and manual approach on this. Indexing of wikis will come automatically as a side-effect as soon as this wiki gains popularity. Probably you will have to think about "banning" large wikifarms (mediawiki, wikia, etc.) since everything is well known about them and they do their own activity monitoring. However focussing on wiki activity and key tasks is the way to go. +1 for this approach. Cheers --[[kgh]] (talk) 08:22, December 29, 2012 (CST)

Splitting into wiki farms

Heiya Jamie, one thing which might be good to do is to split up the stats for wiki farms as they do at wikistats (s23). Putting an emphasis on wikis of one wikifarm may distort the interpretation of the data displayed. Just an idea for the improvement tracker. :) Cheers --[[kgh]] (talk) 07:14, December 29, 2012 (CST)

That is part of what I see using the Tags for. For example, I use my name as a tag for the wikis that I run. Right now this is only used for navigation, but I could see rolling up usage information across the tags in the future. Similarly I tagged Wikimedia run wikis so they can be bundled. Aggregating stats and information across those tags is a future task. Right now the time series data stored outside of the wiki doesn't know about tags. It would be straightforward though to have tags aggregate info just like website_id (example) does now.
Probably I just picked the only wiki here without tags to see how you do it. Sounds reasonable to me. :) --[[kgh]] (talk) 08:24, December 29, 2012 (CST)

Changes to templates and forms

Heiya, I have some minor changes to templates and forms. Just revert the ones you do not like. Cheers --[[kgh]] (talk) 19:08, December 29, 2012 (CST)

All looks awesome! Thanks for contributing!!! Thingles (talk) 06:15, December 30, 2012 (CST)

I tricked myself

Heiya Jamie, I tricked myself and added the extension to the wrong namespace. Thus [[Sudo]] may go. Senior moments I guess. ;) Cheers --[[kgh]] (talk) 05:58, December 30, 2012 (CST)

No worries. I deleted that page and I see you updated the Form:Extension to be smart about the namespace. Thanks! Thingles (talk) 06:12, December 30, 2012 (CST)
The Extension pages themselves should always get created automatically via Property:Has extension and the use Creates pages with form property but it seems that it doesn't always happen. :-\ Thingles (talk) 06:15, December 30, 2012 (CST)
Thank you for deleting. Automatic creation is great, but currently one has to look through them anyway to add the URL and the current version. Updating this will probably be tricky thing. Cheers --[[kgh]] (talk) 06:33, December 30, 2012 (CST)
My hope is to have a bot do that work, but that is far down the list and will still be prone to problems since the Extension information on the MediaWiki site is often out-of-date as well. Thingles (talk) 06:41, December 30, 2012 (CST)
That's indeed true. Perhaps you can tell the bot to throw a message in case it detects an newer version during data retrieval than the one provided at the extension page? This will be of help for the time being, I guess. --[[kgh]] (talk) 06:50, December 30, 2012 (CST)
That's a great idea! Just have Bumble Bee post to the talk pages indicating that the extension appears to have new versions. Nice. Thingles (talk) 07:02, December 30, 2012 (CST)
Cool. :) --[[kgh]] (talk) 07:07, December 30, 2012 (CST)
I scanned Semantic Forms source code to figure out how it attributes automatic page creation and found that as of version 2.5 the variable $sfgAutoCreateUser can be used to override (also mentioned in release notes). I've now set this to User:Bumble Bee so going forward auto-created Extension pages should be associated with the bot, and not me. I also updated mw:Extension:Semantic Forms/Linking to forms. Thingles (talk) 20:35, 18 January 2013 (UTC)

Tags for extensions

Heiya Jamie, what about adding tags to extensions, too? Programmers names could be added and extension groups according to Special:Version (semantic, spam, media, special, parser, other, etc.) Cheers --[[kgh]] (talk) 06:52, December 30, 2012 (CST)

That sounds like a great idea. Should they be in a different property? I'm thinking Property:Has extension tag so that there is a different "tag space" for extensions. Thingles (talk) 07:03, December 30, 2012 (CST)
Definitely. I am just thinking of having not a common include all property, but one for Programmers [[Property:Has programmer]] possibly together with a "Programmer" namespace and one for the group [[Property:Has extension group]] with allowed values. Cheers --[[kgh]] (talk) 07:12, December 30, 2012 (CST)
I would just go with one [[Property:Has extension tag]] for now and see where it goes. A separate developer/programmer property feels redundant to the MediaWiki extension repository. Thingles (talk) 07:15, December 30, 2012 (CST)
Sure, fair enough. --[[kgh]] (talk) 07:26, December 30, 2012 (CST)
Up and running. :) --[[kgh]] (talk) 08:19, December 30, 2012 (CST)
This is awesome by the way! Thingles (talk) 14:32, December 31, 2012 (CST)

Done by Kghbln on 12/30/2013
Create the possibility to add tags to extensions.

I am amazed ...

... to see that just the 100 something websites which have been added here already use about 15 % of all MediaWiki extensions. I would not have expected to see this variety. --[[kgh]] (talk) 14:10, December 30, 2012 (CST)

Wow! That is very cool to know. I personally love to be able to see the versions that are being used. Something I'm going to add to the roadmap is snapshotting the extension/version count each day, so we can see how long it takes for new versions of things to roll out. Thingles (talk) 14:38, December 30, 2012 (CST)
There are about 2200 extensions published on mw.o and I estimate that another third is located on various other places around the web. So its 7.5 % and not 15 %. Ouch.
550+ extensions already from just around 125 websites. Wow. --[[kgh]] (talk) 04:49, January 8, 2013 (CST)
From a devolopers point of view it it probably as interesting to know how long it takes until sysadmins upgrade their MediaWiki, too. I would also add this to the roadmap. Talking about the roadmap: It would be cool if the statistics on PHP versions used could be aggregated by version and not by version and variety, i.e. one bar for 5.3.2 and 5.3.2-lenny-foo. Cheers --[[kgh]] (talk) 15:03, December 30, 2012 (CST)
I created a new template Get simple version number that will take the version just left of the - and separated the properties. There are now properties for the version as well as secondary properties for the version details. This is a very nice change. Thanks for suggesting it! Thingles (talk) 14:32, December 31, 2012 (CST)
Great, that's very smart how you did it. I can definitely learn from you here and will do so. I appreciate this. :)
Thanks for the complement! :-) That template is a simplified version of one I made to get the hostname from a URL. Thingles (talk) 07:58, January 1, 2013 (CST)

Handle Versions Better...

(This topic was split off of "I am amazed ..." above)

Were is still one suggestion which might need some voodoo. I order to get a decent sorting some internal padding will have to be added. So far the sorting is 1, 10, 2, etc. Thus 01, 02, ..., 10 would be cool. Perhaps this is some additional requirement to the request to get decent number formatting to SMW. O_o Cheers --[[kgh]] (talk) 06:21, January 1, 2013 (CST)

The sorting of versions is still a problem. The right way to do that is to store three separate number properties: major version, minor version, fix version. I could easily take the output of the simple version, split it into an array using the . and then shove them into three new properties. Then sort on those. That is likely the real/right way to do it but I'm hesitant to add 9 new properties (MediaWiki, PHP and Database). But I probably shouldn't worry about it. Thoughts? Thingles (talk) 07:58, January 1, 2013 (CST)
I would not worry about having too many properties. With the new SQL-store in place performance should not suffer from this as it would have before. You could even create a separate table for the properties as soon as they hit a substantial number of values. I also think this is quiet a common usecase. I remember people on the mailing list asking to do this with IP addresses. This could even be added to the SMW community wiki as a tip or to the mediawikicookbook though it is currently done or even gone. I should add your website to my favourites. :) --[[kgh]] (talk) 08:26, January 1, 2013 (CST)

Done by Thingles on 1/1/2013
Create new properties to support sorting of versions.
Done for PHP and Database. Still need to do for MediaWiki version. Thingles (talk) 09:03, January 1, 2013 (CST)
 Has PHP VersionHas PHP Version detailedHas PHP major versionHas PHP minor versionHas PHP maintenance version
.NET Micro Framework Wiki
.hr domene
/usr/space Wiki7.0.33-0+deb9u1070
001 Wiki
007 Fanon Wiki7.3.2373
007Film. Wiki
… further results

To Do for Thingles  on 1/1/2013
More robust version properties for MediaWiki, similar to PHP and Database versions.
Awesome. I just changed the stats page. This is really cool. It is not as urgent for MediaWiki and extensions since both rarely go beyond 9. However, I expect that 1.19.x will be the first in a long time to do just that. --[[kgh]] (talk) 14:32, January 1, 2013 (CST)


Heiya, what about this extension? I think it could be useful for e.g. "Main page" and "Extension:Main page" Cheers --[[kgh]] (talk) 05:57, December 31, 2012 (CST)

Maybe, but I wouldn't mind allowing the Main Page to be cached though. In general the current layout will only work so long. Now at over 100 sites there are probably too many. I started a topic on Talk:Main Page to explore what really belongs on the main page. Thingles (talk) 14:34, December 31, 2012 (CST)
I'm a little worried about overusing this extension, but I did go ahead and install it (see Special:Version) and activated it on the Statistics and Semantic statistics pages. Thingles (talk) 07:46, January 3, 2013 (CST)
This is indeed only an extension for selected pages. I do not think that it will be overused. --[[kgh]] (talk) 10:00, January 3, 2013 (CST)

Done by Thingles on 1/3/2013
Added MagicNoCache extension so frequently used pages can be marked as not cached.

Short descriptions for extensions

I think we will have to add short descriptions to the extensions as well in order to add additional value. Just having the name, the current version and a link to it does not seem enough to me. --[[kgh]] (talk) 06:37, January 1, 2013 (CST)

I've been struggling myself with the fact that I'm throwing away all this data that I get when I pull extensions. This is the JSON block I get back
	"query": {
		"extensions": [
				"type": "parserhook",
				"name": "SyntaxHighlight",
				"description": "Provides syntax highlighting using [http:\/\/\/highlighter\/ GeSHi Highlighter]",
				"descriptionmsg": "syntaxhighlight-desc",
				"author": "Brion Vibber, Tim Starling, Rob Church, Niklas Laxstr\u00f6m",
				"url": "http:\/\/\/wiki\/Extension:SyntaxHighlight_GeSHi"
				"type": "parserhook",
				"name": "ParserFunctions",
				"description": "Enhance parser with logical functions",
				"descriptionmsg": "pfunc_desc",
				"author": "Tim Starling, Robert Rohde, Ross McClure, Juraj Simlovic",
				"url": "http:\/\/\/wiki\/Extension:ParserFunctions",
				"version": "1.3.0"
				"type": "parserhook",
				"name": "CategoryTree",
				"description": "Dynamically navigate the category structure",
				"descriptionmsg": "categorytree-desc",
				"author": "Daniel Kinzler",
				"url": "http:\/\/\/wiki\/Extension:CategoryTree"
When I record the extension for the website I'm only capturing the stuff unique to the website, name and version (if provided).
I could easily modify the bot to write all these fields in the extension pages for each website. Or, more difficult, is to have the bot modify the Extension pages in the extension namespace with this information. But that introduces complications like what if author value changes from 1.0 to 1.1 and both are in use. The bot needs to have logic to know that 1.1 > 1.0 and write that. Complicated.
Maybe for now I should just modify the bot to write all data in the extension subpage, and thus into the extension subobjects and then see what we can do in the templates? That means we would have 100 descriptions for Extension:ParserFunctions that probably are all the same, but maybe not. Then we decide what we want to pull into the Extension:ParserFunctions page. Thingles (talk) 07:39, January 1, 2013 (CST)
Hmm, actually this only has to be done once for each extension upon creation. Only versions change frequently and sometimes the author. Perhaps it is possible to do this once, add the data to the page and check e.g. quarterly if there was a change or so. Whatever appears more easy to implement I suppose. Cheers --[[kgh]] (talk) 08:42, January 1, 2013 (CST)
Should I continue to gradually add the information manually? Cheers --[[kgh]] (talk) 13:55, January 3, 2013 (CST)
I have just expanded the form and template to also allow entering a description. We should have it no matter if it may be filled automatically or not. Cheers --[[kgh]] (talk) 13:44, January 8, 2013 (CST) PS I did some formatting. Looks ok to me now, e.g. Semantic MediaWiki --[[kgh]] (talk) 14:18, January 8, 2013 (CST)

Database error on saving a new wiki

Heiya Jamie, the External Data extension causes some troubles:

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

    (SQL query hidden)

from within function "EDUtils::searchDB". Database returned error "1064: You have an error in your SQL syntax; ...

Did you change something there? Cheers --[[kgh]] (talk) 07:03, January 1, 2013 (CST)

My bad. Fixed (diff) by checking for existence of {{PAGEID}} before calling the Template:Website indexes. {{PAGEID}} is an odd magic word since it doesn't exist until after the first save. So, if you use it, you have to conditionally check it if you have dependencies on it. I used {{PAGEID}} as my external key for the bots and the Apiary database to work with the wiki.
I could have figured this out myself. Yes, this magic word is a bit odd, but this is within it's nature. To tell you the truth: I asked for this on bugzilla about two years ago and I was very surprised that they actually added it. --[[kgh]] (talk) 08:29, January 1, 2013 (CST)
I'm glad you asked for it and that they put it in. It's actually a very very critical magic word for cases where wikis have to interact with other systems or applications. Thingles (talk) 08:42, January 1, 2013 (CST)
This was the very reason why I asked for this. At the beginning there was quite some opposition to it. Luckily a story of history past. --[[kgh]] (talk) 08:44, January 1, 2013 (CST)

Identification of Bumble Bee

Heiya Jamie,

I am just checking my access logs and found, not surprisingly at all, your bot:

IP - - [31/Dec/2012:23:45:07 +0100] "GET /api.php?... HTTP/1.1" 200 135 "-" "Python-urllib/2.7"
IP - - [31/Dec/2012:23:45:07 +0100] "GET /api.php?... HTTP/1.1" 200 101 "-" "Python-urllib/2.7"

Is it possible to provide identification for it, e.g.

... "Python-urllib/2.7" (Bumble Bee; +

This would be an extra service for server admins and prevent it from being blocked just because it is unknown.

Cheers --[[kgh]] (talk) 17:57, January 1, 2013 (CST)

Great idea! I've changed it and you should be seeing Python-urllib/2.7 (WikiApiary; Bumble Bee; + in your logs. Can you verify? Thingles (talk) 18:36, January 1, 2013 (CST)
Affirmative. Bumble Bee identifies itself as described. Thank you for doing this. Cheers --[[kgh]] (talk) 03:25, January 2, 2013 (CST)

Done by Thingles on 1/1/2013
Specified user-agent for Bumble Bee bots so that server admins can tell who is talking to them.

Done by Thingles on 1/1/2013
Fixed bug in that caused exception if Semantic MediaWiki stats were asked for but not returned. Now skips and continues.

Moving up the ladder

Thank you for the promotion to Operator. :) I wonder why there is no spam activity on this wiki yet. Once I installed a new wiki and within 10 minutes [!sic] I had the first spam entry. Those were the days. ;) --[[kgh]] (talk) 14:05, January 7, 2013 (CST)

Automatic creation of extension's pages

Heiya Jamie, I think that this is done automatically via your account for all missing extensions of a website as soon as somebody created the first of the missing ones. This is at least my observation. Cheers --[[kgh]] (talk) 04:46, January 8, 2013 (CST)

Oh yeah, those pages are created automagically using Semantic Forms "Creates pages with form" property on Property:Has extension. I wish I could specify another account as the user for those creates. I couldn't find a setting for that. Thingles (talk) 09:57, January 8, 2013 (CST)
Ah, this functionality slipped my mind. I now remember to have read about it before. This is a great usecase for this. Why does it pick you user name anyway? Because you set it up with your account. Perhaps creating another account (bot) and redoing this will switch over to the new one to be used. --[[kgh]] (talk) 14:25, January 9, 2013 (CST)
Bumble Bee took over today. How did you do it? --[[kgh]] (talk) 09:24, 23 January 2013 (UTC)
Nothing I did. You mean it filled out a bunch more of the red links? It seems like Semantic Forms picks those things up after Bumble Bee makes a number of changes to Extension subpages. It's really unclear to me when it does that work though, since the timing of those page creates is a bit delayed from when the pages get updated. Very voodoo. :-) I wonder how many sites use that Creates pages capability to provide a solid test bed. Thingles (talk) 13:02, 23 January 2013 (UTC)
No you did not, I just noticed that Bumble Bee did the job today instead of Thingles. Mostly once I created one extension's page for a website, the autocreation kicks in for the extension associated with it. Not in every website's case though. Yeah, it is a bit voodoo but better than doing everything manually. :) --[[kgh]] (talk) 15:51, 23 January 2013 (UTC)
Oh, I found that setting and changed it a few days ago. See User_talk:Thingles#I_tricked_myself. Also updated the docs. Thingles (talk) 16:49, 23 January 2013 (UTC)
Yay! Good to know. This post slipped my attention. --[[kgh]] (talk) 20:48, 23 January 2013 (UTC)
Easy to happen, there is a lot here. I'm going to archive this talk page at the end of January. Thingles (talk) 02:03, 24 January 2013 (UTC)
Tidying up can probably also be done with a bot. e.g. Archive Bee. On Wikipedia they have bots which automatically archive sections which are tagged as done, etc. I guess they also provide the python scripts. Cheers --[[kgh]] (talk) 12:24, 24 January 2013 (UTC)

Farm:Main page

I just created a proto page for this too. Cheers --[[kgh]] (talk) 15:11, January 9, 2013 (CST)

Thanks! Thingles (talk) 21:58, January 11, 2013 (CST)

Pages to go :)

[[Template:Farm by Tag]] and [[Extension:Biowikifarm]] may go the happy hunting grounds. Thank you --[[kgh]] (talk) 16:51, January 10, 2013 (CST)

Thank you! --[[kgh]] (talk) 06:06, January 11, 2013 (CST)
 :-) Thingles (talk) 21:58, January 11, 2013 (CST)

Another one to go: [[Farm:Wiki-Family]]. Thank you an cheers --[[kgh]] (talk) 17:33, 25 January 2013 (UTC)

Done! Thingles (talk) 18:08, 25 January 2013 (UTC)
Thank you. --[[kgh]] (talk) 18:12, 25 January 2013 (UTC)

Now it's a file [[:File:Wikipedia Logo.png]]. Thank you and cheers --[[kgh]] (talk) 18:34, 28 January 2013 (UTC)

Done! Is there any reason that I don't just add deletion rights to Project:editors? Thingles (talk) 04:46, 29 January 2013 (UTC)
Thank you. Hmm... no, because I would just assume good faith here since it may still be revisited. --[[kgh]] (talk) 09:32, 29 January 2013 (UTC)

[[:File:Serenity Logo.jpg]] may go. --[[kgh]] (talk) 18:52, 10 February 2013 (UTC)

[[Erich Steiger2]], [[Wiki-Family/General]], [[Wiki-Family/Extensions]] and [[Wiki-Family]] too. --[[kgh]] (talk) 13:17, 13 February 2013 (UTC)

All deleted. Also, I finally added delete permission to project:Editors so you can do these directly. Thingles (talk) 16:19, 14 February 2013 (UTC)
Great and thank you for the delete permissen. You can still block me if I am doing to much strange things. ;) --[[kgh]] (talk) 16:41, 14 February 2013 (UTC)


I just wanted to say that I really like the boilerplate logo you created for farms. :) Still this is not the only reason why I am posting: What about adding a statistics section with php, mysql and mw versions to the main farm pages? I guess this would be quiet nice to have, too. --[[kgh]] (talk) 10:38, January 13, 2013 (CST)


This seems to be our current mother of semantic wikis when it comes to property values. I remember to have seen one with over 40,000,000 property values about two years ago, but I cannot remember which site it was. Probably they are well over 50,000,000 by now. Talk about pushing the limits. :) --[[kgh]] (talk) 10:41, January 13, 2013 (CST)

Well, I found it: Metacafe Allow about 2 minutes for the page to show. Sadly they are still on SMW 1.4.2 "This wiki contains 57,372,859 property values for a total of 642 different properties. 545 properties have an own page, and the intended datatype is specified for 527 of those..." Cheers --[[kgh]] (talk) 18:12, January 13, 2013 (CST)

Response time

This is an excellent extra feature. People will love it and admins will hate it. Even I knew about what is there black on white for my pages I did not like to seen this. ;) Cool performance on your farm. Good setup or a strong machine or both? Yeah, probably both. --[[kgh]] (talk) 10:45, January 13, 2013 (CST)


Unbelievable. This is actually running MediaWiki. You ought to see this. Cheers --[[kgh]] (talk) 12:29, 14 January 2013 (UTC)

Wow, that is awesome. That's one of the most interesting skins I've ever seen. Bravo! Thingles (talk) 13:47, 14 January 2013 (UTC)
I knew you would appreciate this. :) I do not know if it will be out of scope, but having a "Wiki of the Month" featuring outstanding wikis from the collection might be an idea too. However, I am not thinking of long descriptions but just of a direct link from main page. Another way would be to do a RandomSelection of links to cool wikis, to be included on main page. Hmm..., I think I like this even better. --[[kgh]] (talk) 18:51, 14 January 2013 (UTC)
Would be trivial to add a tag "cool" and pick a random wiki with tag cool to highlight. Thingles (talk) 19:17, 14 January 2013 (UTC)
This is an less complicated option, too. Yeah, we could go for it in case you like this idea. Will probably be an "operators" thing. Cheers --[[kgh]] (talk) 19:58, 14 January 2013 (UTC)
Jazzorakel‎‎ is another nice showcase wiki. I would definitively opt for having it. Cheers --[[kgh]] (talk) 23:41, 18 January 2013 (UTC)
We should now decide on having this or not before working through the import? It should be an operator privilege to set this. --[[kgh]] (talk) 15:31, 26 January 2013 (UTC)
Creating new topic #Featured website

Featured website

So the objective would be to make a way to highlight some subset of sites as special. We could have another checkbox for this and associated property, something like Is featured website. That also could be a number, like Feature website value that could be use to weight randomness in a featured Web site content area? A number between 1 and 100 could be used allowing sites with a larger number to be highlighted more often? That would be purely subjective but would be nice to introduce some variation? That would also extend to some idea of operators "upvoting" and "downvoting" the featured websites. Thoughts? Thingles (talk) 16:16, 26 January 2013 (UTC) PS: I'll be offline for a while hanging with the family.

This system would be subjective to a great extent. However, it should probably be possible for the community too to suggest an wiki to be featured, too. Since you are asking, I would also opt for weighing randomness to provide more flexibility later on. This flexibility could consider votings and/or the age of the last revamp of a wiki and/or size of a wiki and/or different content areas and/or semanticness and/or other factors that may come along the way. Yeah, this is the way to go for the Has featured website value. --[[kgh]] (talk) 17:01, 26 January 2013 (UTC)
This is now added. Property:Has featured website vote created. Template:Website has been modified to display and set it, along with an autoedit button to increment the vote. Form:Website also added to display the value. I created a separate "WikiApiary Operators" table to to the form that would be nice to style differently to show that is operator only stuff. Given the idea of using multiple votes for weighting I expect I'll just put together a bot that pulls all sites with votes and does a weighted random pick once a day. That bot can put the pick for that day someplace like [[WikiApiary:Site of the day]] and then another template that displays it on the front page can transclude to get the pick. At least that's what makes sense to me. Thingles (talk) 13:52, 27 January 2013 (UTC)
This is great. I just changed the for a bit to make it visually more striking. It was a bit confusing at first sight. Having a result format that considers this, a bit similar to the value rank would be cool. However to have Feature Bee is nice too. To do it via an embedded template is the way to go. Thank you for doing this and thus adding more value. --[[kgh]] (talk) 12:29, 28 January 2013 (UTC)


Things like this seem to happen too. Two extensions with the same name. I am going to author a rename for one of them. Hopefully it goes through. Cheers --[[kgh]] (talk) 18:46, 14 January 2013 (UTC)

I don't think Semantic Forms created that page as a duplicate. Extension:DynamicPageList2 came from Wikicafe. I looked on their Special:Version page and sure enough, they have one named with a 2 appended. Not sure what to do with that, my instinct would be to leave it.
This does highlight something I think I need to add to User:Bumble Bee, which is support for blacklisting extensions. I'm thinking of a page that contains regex patterns that Bumble Bee will look at and ignore extensions with names that match a regex. Thoughts? Thingles (talk) 19:16, 14 January 2013 (UTC)
There is not need to blacklist the extension since both are valid extensions in active maintenance. They just have the same name. Thus I do not think that is is Bumble Bee's problem. At one stage it was decided to rename Intersection to DynamicPageList "(WM)" which is the root of the problem. I just changed the name an hope it gets through. Still it would take some time for the change to roll in here since the wikis using it will have to be updated, but time is going to heal this. --[[kgh]] (talk) 19:58, 14 January 2013 (UTC) PS Everything else may serve as plan b. PPS DynamicPageList2 was the former name of the now DynamicPageList "(tp)" O_o. After all the upgrading is done and the extension "DynamicPageList2" is empty we can easily make a redirect to "DynamicPageList (tp)" --[[kgh]] (talk) 22:33, 14 January 2013 (UTC)


I think it is also time to change to QuestyCAPTCHA which has so far proven to be the best solution. Cheers --[[kgh]] (talk) 17:38, 16 January 2013 (UTC)

Noted. Will put this on my extension list. Thingles (talk) 05:46, 17 January 2013 (UTC)
The worst part is inventing questions. I will send some ideas to you which seem to do the trick. --[[kgh]] (talk) 10:03, 17 January 2013 (UTC)
Well, this is now done. Unfortunately, it seems that ConfirmEdit/MathCaptcha and the Math extension had a disagreement and started blowing up. :-( Bug 46132 has been filed for this. I've switched over to Questy but I've only got one question setup right now. I also created User:Jthingelstad as a regular user account so I can test this stuff.

Done by Thingles on 3/14/2013
Change to QuestyCAPTCHA for registrations.


Heiya Jamie, what about having a Welcome Bee welcoming users and telling them some basics about the Apiary. Cheers --[[kgh]] (talk) 18:46, 16 January 2013 (UTC)

That extension caused me some problems a while back, and in hindsight I'm not sure it was the extension or something else. It would be a good idea. If you wanted to take a whack at writing the welcome copy I could get the extension in this week. Thingles (talk) 05:45, 17 January 2013 (UTC)
I use it on CAcert in Berlin smoothly and is using it without problems, too. I believe that this extension is also sustainable when it comes to maintenance. I will think of a nice text for Welcome Bee. --[[kgh]] (talk) 10:02, 17 January 2013 (UTC)
Done. Installed. See Special:Version. It's fun when WikiApiary helps me do stuff on WikiApiary. I can see that 9 sites are running this extension and that gives me confidence. That's the goal! :-) Thingles (talk) 13:41, 17 January 2013 (UTC)
 :) Yet alone that is using it provides stability in maintenance. It seems very unlikely that they will abandon usage of the extension. --[[kgh]] (talk) 16:49, 17 January 2013 (UTC)
Do not forget to add Welcome Bee to MediaWiki:Newusermessage-editor, since "New user message" is not very appealing. To not forget to make it a bot, too. :) Cheers --[[kgh]] (talk) 21:29, 17 January 2013 (UTC) PS I also wrote a welcoming text. Feel free to revise it.
Thanks for the reminder! Just filled out MediaWiki:Newusermessage-editor and set it to the existing User:Notify Bee account since that is what will be sending future email notifications anyways. It's a bot as well. I also tweaked out Template:Welcome. Awesome! Thingles (talk) 13:13, 18 January 2013 (UTC)

Done by Thingles on 1/17/2013
Installed New User Message extension.


Heiya and hmm..., I would rather go for pages instead of articles. The classification of an article is rather arbitrary and on most wikis that what was added to the main namespace. I have seen a lot of wikis where you have pages with article quality in other, mostly self-defined namespaces, too. I doubt that they set $wgContentNamespaces and did an php InitStats.php afterwards. From an sysadmins point of view the total number of pages will probably the better shot anyway. --[[kgh]] (talk) 22:21, 16 January 2013 (UTC)

Yeah, I agree. Changed. Thingles (talk) 22:30, 16 January 2013 (UTC)
I'm curious, in graphs where you show Articles and Pages, would you also put the emphasis on Pages? I've been adding slight emphasis to Articles over Pages, but maybe it should be the opposite way? Thingles (talk) 05:56, 17 January 2013 (UTC)
Yes, because the definition of an article is wiki specific. This qualitative differentiation is probably a remnant or special case with regard to e.g. Wikipedia. You cannot trick the number of total pages or better said, it is clear what is meant with that - no further discussion needed there. When it comes to activity, an emphasis of WikiApiary, all namespaces seem relevant to me (talk pages in general are interesting too), some obviously to a lesser degree because of their nature (MediaWiki:). Another thing: When it comes to edits on an wiki, all edits are counted anyway, so having a clear path here ... --[[kgh]] (talk) 09:59, 17 January 2013 (UTC)
Excellent. I changed the Widget:Website graphs to give visual priority to pages over articles. I need to add images in their too, but sort of need a 3rd y-axis which dygraph won't do. Will figure that out later. Thingles (talk) 13:36, 17 January 2013 (UTC)
Great. I think images a different pair of shoes. I do not exactly know if all images with pages are counted or just the number of uploaded images. I presume the latter is the case. If so, images do not really fit into the articles and pages graph and therefore an additional graph is needed. So far we have pages and articles in the graph. I would rather add a line for other pages there which is the subtraction of pages and articles to make this graph a bit more clear. Pages on one y-axis and articles and other pages on the second y-axis. By the way: It would be cool if you could colour the description of the y-axis in accordance with the lines. --[[kgh]] (talk) 16:47, 17 January 2013 (UTC)
Continuing this, I have a calculated index on all websites for Property:Has statistic article change. By the thinking above, this should also be switched from articles to pages. Agree? Thingles (talk) 13:59, 18 January 2013 (UTC)
Yes. --[[kgh]] (talk) 23:21, 18 January 2013 (UTC)
Whew, minor surgery done. Property:Has statistic article change is no more, long live Property:Has statistic pages change! Thingles (talk) 05:45, 19 January 2013 (UTC)
Okay, even more articles versus pages. I'm noting the entirety of Main Page is designated by article. That should be pages too yes? If only to be consistent? Thingles (talk) 18:59, 25 January 2013 (UTC)
Yes, this should be pages, too. Not just for consistency, I think. --[[kgh]] (talk) 19:10, 25 January 2013 (UTC) PS I would not move properties but creating new ones and deleting the old ones. Something is strange about moved properties.
Not moving properties is always a best practice it seems. :-) This wont require any property changes, they are already both stored as separate. Just changes to the ask criteria. Thingles (talk) 19:16, 25 January 2013 (UTC)
I'm going to change the groupings to be (partly) concept based as well and will do the switch at the same time. Thingles (talk) 19:11, 25 January 2013 (UTC)

Done by Thingles on 1/18/2013
Changed the article change index to be based on pages instead.

Automation of extension and wiki discovery

I am wondering to what extend the process of adding new wikis and new extensions here, as well as updating info for the extensions, is automated --Jeroen De Dauw (talk) 10:59, 18 January 2013 (UTC)

Websites and Farms are entered by users using Form:Website and Form:Farm respectively. They are not automated. It would be pretty easy to automate the creation of websites with a spidering bot, but I don't see this (at least at this moment) as the mission of WikiApiary. For example, I don't plan on loading the bulk file of Farm:Wikia sites. They monitor and maintain their wikis professionally. The original thought was that the hundreds (thousands?) of people maintaining their own MediaWiki sites would benefit from the information here, as well as receive helpful notifications from WikiApiary to make their wikis run better. I put in the first 100 websites so I had things to develop with. I also put in a lot of Farm:Wikimedia just because I think it would be odd to not include them in a project like WikiApiary. (It may be illogical to think that Farm:Wikimedia should be included but not think Farm:Wikia should. Not sure.) With that said, I do think that bulk loading all Semantic MediaWiki websites would be fine. Or bulk loading other independent wiki farms. In fact, I've documented the process for that at WikiApiary:Bulk import websites.
(In brief, if someone wanted to write their own bot to add hundreds or thousands of their own wiki sites to WikiApiary I would be fine with that. And pretty stoked about it. :-P)
Extensions are very automated, as is all information about the websites (Property:Has HTTP server, Property:Has PHP Version, Property:Has MediaWiki version, etc.). You'll note that each website has two subpages, WikiApiary/General and WikiApiary/Extensions. These are completed automated via User:Bumble Bee. Bumble Bee is the same bot that collects statistics data as well. The general site information and extension data is collected once every 24 hours. Extension pages in the extension namespace are auto created via Semantic Forms Creates pages with property using Property:Has extension. However, [[kgh]] has done a ton of work in the extension namespace adding descriptions, URL, current version information, etc. But, for a default page to exist no human is required. Unfortunately those pages are created with me as the user, which I wish I could change but haven't figured out a way. Also, notably there are times when Semantic Forms just doesn't want to create some of those pages and I note that [[kgh]] usually fills in the 'redlink gap".
I have plans for more bots as well. The code for the rewrite of Bumble Bee and future WikiApiary bots is on the WikiApiary repository on Github. The next one I'm most interested in is User:Notify Bee which will send emails to people who've requested them with helpful information.
Long answer, but wanted to give you the details. Thingles (talk) 12:58, 18 January 2013 (UTC)
Thanks for your thorough answer :) --Jeroen De Dauw (talk) 18:44, 18 January 2013 (UTC)

Precision for users and edits

For wikis that have few users or few edits, the graphs have scales with float numbers. Perhaps better to limit this to ints if possible. Not vert important I guess --Jeroen De Dauw (talk) 11:02, 18 January 2013 (UTC)

I'm having a hard time finding a site that is showing floats. Which pages are you seeing this on? Thingles (talk) 12:44, 18 January 2013 (UTC)
Ahh, found one. User graph for ARIN IPv6 Wiki. Gonna see if I can tweak Widget:Website graphs to avoid this. Thingles (talk) 13:38, 18 January 2013 (UTC)
Okay, not showing decimals doesn't seem to be trivial. I tried the digitsAfterDecimal: 0 option but that was worse since two lines showed up for a value and they seemed to be the same value. It just truncated the decimals. I don't see any other dygraph options that would do it. For now, I'm just going to leave this as is. Thought it might be a trivial fix. Thingles (talk) 13:57, 18 January 2013 (UTC) PS: I did add fills to some of the graphs while poking around, which I think looks a bit nicer.

mutante's wikistats

Hello, I read on kgh's talk that you're already importing some groups from s23. Of course I think you should import them all as soon as possible, ;-) however note that the new location is , where it had some work including the import of several lists by Wikiteam, the most important of which is Pavlo's: if you use the txt list directly note that there are some duplicates and many of them are tracked. mutante's approach is of course quite different from yours, and by the way the repo is open for all to see and patch: I'm extremely curious to see how this wiki goes on, and if you manage to import all the independent MediaWikis I'll surely add a link to the Extension pages from mw:Template:Extension. --Nemo bis (talk) 12:54, 18 January 2013 (UTC)

Indeed I have spent some time looking at that Wikistats 2.0 site. I would feel fine adding everything outside of Farm:Wikia pretty soon. There are some data points that that has that I'm not sure I can get easily, notably "stub ratio". I'll take a look at this soon. I have documented a CSV format to WikiApiary:Bulk import websites, do you happen to know if there is a relatively easy way to get the the list of sites in s23 in this format?
Also, adding a link to the Extension pages from mw:Template:Extension would be absolutely awesome. I completely expect we will have plenty of sites to generate very statistically relevant information for that. I've already found it helpful when working with extensions. It isn't ready yet though, we need to add more polish and more capability to those pages. Also, an idea that I think would be interesting is for mw:Template:Extension to display via an API call to WikiApiary the most commonly installed version of extensions directly on the mw:Template:Extension. Just an idea. Also, I really like how WordPress shows the grid of WordPress version versus Plugin version. Something like that could be done with this data. And, on the Project:Roadmap I plan on snapshotting the version data for extensions nightly so we can get a sense of how long it takes for wiki administrators to get new versions out.
Thank you for checking out the site! Thingles (talk) 13:21, 18 January 2013 (UTC)
What data do you really need for your CSV import? wikistats takes all that from the API (or Special:Statistics if the API is not available), don't you do the same? And if not how do you update? Back to your question: there is the CSV version, currently it works for me.
Yeah, all the data collected by WikiApiary is through API using a bot, that is what generates the data for the graphs. (I don't have anything to read Special:Statistics). The list of sites to collect from is where a CSV can help. I downloaded that file and it has the two columns I would need, namely the wiki URL and URL to api.php. It's going to be very tempting to load those all in. :-) Thingles (talk) 13:53, 18 January 2013 (UTC)
The stub ratio is just the ratio between "good articles" (countable articles) and all pages. See also Wikipedia article depth.
I added Depth calculation to each website, and I also stored it in Property:Has statistic depth so it can be easily queried. Thanks! Thingles (talk) 19:28, 18 January 2013 (UTC)

Done by Thingles on 1/18/2013
Added depth calculation per
Our templates are not able to make API calls, we don't even have interwiki transclusion (scarytransclusion) enabled... Some of your proposal seem to fit with the mw:Extension Matrix project; the code used to update it is on the wiki. --Nemo 13:47, 18 January 2013 (UTC)

Extension:Header Tabs

What about setting $htEditTabLink = false; to avoid having edit links within a template using header tabs and $htRenderSingleTab = true; just in case there will be some use for it in the future? Cheers --[[kgh]] (talk) 13:42, 20 January 2013 (UTC)

Yeah! Thanks for highlighting these settings. That [Edit] link has bugged me all along but not enough to figure out how to make it go away. Both directives have been added to LocalSettings. Thingles (talk) 21:26, 20 January 2013 (UTC)


I just had a look. As a matter of fact the MediaWiki API was introduced with the 1.8 branch. All version prior to this did not have it at all. Still have to find out at which stage data retrieval was possible, too. --[[kgh]] (talk) 14:18, 20 January 2013 (UTC)

Nothing useful before 1.11, as far as I can see: mw:API:Meta#siteinfo_.2F_si. 1.14 or even 1.15 (for language) required to have all the info you seem to be using now. (That's why wikistats must use Special:Statistics?action=raw...) --Nemo 14:44, 20 January 2013 (UTC)
Great, thank you for the info. Hopefully your recent offensive via e-mail helps pushing wikis up to an at least usable version. Cheers --[[kgh]] (talk) 16:32, 20 January 2013 (UTC)
After seeing the use of "Special:Statistics?action=raw" on wikistats I dug into that a bit and considered if there was a place for it on WikiApiary. My initial thought was no, if the wiki hasn't been upgraded in that long we wont put it in here. 100% coverage of wikis isn't one of the (current) objectives of WikiApiary. I put current in parens because I feel a pull towards pulling in more and more so that statistics around wikis are more accurate. However, I'm not going to screen scrape the Special:Version page for extensions so including Special:Statistics?action=raw info would end up skewing extension information. It wouldn't be hard to have another method in User:Bumble Bee that did Special:Statistics collection. I'd have to consider how I would want to indicate that method should be used versus the direct API method. What do you guys think, should WikiApiary support this method? I'm open and would be happy to add it if it should be. Thingles (talk) 21:21, 20 January 2013 (UTC)
I think that WikiApiary has the advantage of being a new thing and of not needing to replace wikistats (which is still alive and wants to go on): mutante's wikistats has a lot of historical data, but this is not a requirement nor a current priority for this wiki I guess. My personal suggestion would be to do the easy part you're good at doing first, i.e. add more API-enabled wikis (there's plenty) and then decide how to move forward; it's up to you.
Just to make very sure Nemo, it's definitely not my intent to try to replace wikistats or displace it. I'm pretty sure the things I'm thinking for WikiApiary are a different focus. I just want to make sure nobody thinks I'm trying to push something else away or am stepping on peoples toes. The genesis for this was my own personal desire to have better graphs of my wiki than what I was getting from Munin, and wishing something would email me when my extensions had upgrades. Things just expand from there. Thingles (talk) 01:27, 21 January 2013 (UTC)
Telling the bot to fetch from "Special:Statistics?action=raw" would perhaps not be that hard to program nor so resource-intensive to run, so it can be considered later. In the list above there are also many wikis using Special:Statistics which actually have an API, it would also possible to run a bot to test them for existence of api.php in the same scriptpath, in one of the most common paths, or at the URL defined by RSD info in HTML source's link rel="EditURI". --Nemo 21:47, 20 January 2013 (UTC)
I believe WikiApiary should not list wikis with versions prior to MW 1.8+. However, the main focus should really be MW 1.14+ (2/2009+) because this is when the extensions kick in, which is an integral part of this wiki. Admittedly I have been adding some of these quaint ones (<MW 1.8.0), mostly because they were saved as bookmarks in my browser. I should pay more attention in future. --[[kgh]] (talk) 23:09, 20 January 2013 (UTC) PS Everything else may probably be sorted out at some point in the future. And yes, there is wikistats, too.
Thank you for the feedback guys and I agree. No reason to do work, especially so early in the project, to support wikis not getting upgrades. Thanks. Thingles (talk) 01:17, 21 January 2013 (UTC)

This issue recently came up again with an email question from Mark Hershberger. And then, just a couple days later, discovered that to collect data for the 3,000+ wikis hosted on Farm:Wikkii Special:Statistics will have to be used since the API is disabled on all of them. So, I've started down this road.

Collection errors on talk pages

Bumble Bee adds a message every time it encounters an error, so up to every 15 minutes. Just 2 hours of outage will e.g. trigger 8 messages. It should somehow check if the problem encountered is the same as the one before. Perhaps an easier way is to stop reporting for 24 hours in case an error message was thrown. Thus the admin would get a reminder judz once a day which I believe is sufficient. Cheers --[[kgh]] (talk) 00:51, 21 January 2013 (UTC)

Yeah, I tried to slap it in easily which was to do it without knowing the state of the interface before hand. I've disabled the logging to Talk pages for now until I can put something more robust in place. Probably just store the state on the wiki, record a property for the timestamp of the last message or something. Thingles (talk) 01:16, 21 January 2013 (UTC)

Bulk import of websites

Heiya Jamie, I just saw that a whole new lot of wikis were added. In case you do this for non-farm wikis please only do 20 a day maximum because it takes some time for me to work my way through (logos, descriptions, missing extension pages, etc.) Cheers --[[kgh]] (talk) 20:43, 23 January 2013 (UTC)

That sounds good. It hit me after you posted this that I forgot to put a description field in the CSV that I imported. I mainly did this one as a learning experience. See how it worked out. I've already noted something I wasn't expecting, a bunch of those 100 Wikibooks sites are completely inactive, and in fact have a message that they are locked on their home page. So, it hit me that User:Audit Bee should exist. Audit Bee will check wikis once a week, and will warn on the talk page for wikis that have had no edits in a week. After 8 weeks of warning, Audit Bee will mark the site as not active. This will be a good way to make sure that WikiApiary isn't spending valuable resources looking at completely abandoned websites. So, those locked Wikibooks sites will be monitored for 8 weeks and then marked inactive as a test case. As will any other wikis which are eventually abandoned. Thingles (talk) 02:01, 24 January 2013 (UTC)
I worked out allrighty :) and + for the Audit Bee. Perhaps 12 instead of 8 weeks would be better to prevent long holidays causing deactivation. Cheers --[[kgh]] (talk) 12:34, 24 January 2013 (UTC) PS You could do a reimport including the description without problems. It is just another revision.
I was originally thinking 12, then thought 8. Your reaction triple-confirms it should be 12. :-) The Talk message will also instruct the person to make an edit of any kind on the wiki in question to restart the timer. Thingles (talk) 19:14, 25 January 2013 (UTC)
If all things in life were this easy to sort out. :) Hey, why don't you create Edit Bee, to work on these messages on foreign wikis? :D --[[kgh]] (talk) 19:45, 25 January 2013 (UTC)
I’m thinking of loading all the Wikipedia sites this weekend. Next bundle. Thingles (talk) 00:33, 26 January 2013 (UTC)
What about focusing on non-wikimedia-sites? I am curious to see what others do an utilise on their wikis, especially exension-wise. --[[kgh]] (talk) 00:41, 26 January 2013 (UTC)
maybe both. Loading 1 or 2 more wikimedia farms is easy since they are in effect clones. I'll see about taking the 4000 or so independent wikis and getting a file ready to break into groups of 10 to import in small bites. Thingles (talk) 00:55, 26 January 2013 (UTC)
What about Wikiversity and Wikinews for the start. Huuuuu, 4000+ wikis in 10 bits at a time. Hmm ... Make it 20. I will try to keep up. --[[kgh]] (talk) 01:02, 26 January 2013 (UTC)
Wikinews and Wikiversity are now completely populated. I didn't even bother doing an upload for Wikiversity, there were only a few not in already. Wikinews added a bit more than 20 more sites. Thingles (talk) 02:21, 26 January 2013 (UTC)
Wikiquote and Wiktionary are ready for take off when you are ready. :) --[[kgh]] (talk) 18:06, 26 January 2013 (UTC)
You taunt me! :-) Thingles (talk) 18:14, 26 January 2013 (UTC)
 :) And I thought that you were off with you kiddies. --[[kgh]] (talk) 18:19, 26 January 2013 (UTC)
Just a quick stop by the talk page. :-P Thingles (talk) 20:03, 26 January 2013 (UTC)
So, this was indeed only a quick stop. ;) Cheers --[[kgh]] (talk) 01:20, 2 February 2013 (UTC)

Text replace

Heiya Jamie, could you please do a text replace from " - Wiki-Family" to "ZUM - Wiki-Family (school)". Thank you --[[kgh]] (talk) 17:32, 25 January 2013 (UTC)

Actually, I went ahead and added replacetext permission to WikiApiary:Editors. You are good to go with that change yourself. Thingles (talk) 18:07, 25 January 2013 (UTC)
Cool, will do! Thank you. --[[kgh]] (talk) 18:09, 25 January 2013 (UTC)

Duplicate wikis

The new warning is very good. Still this only covers a fraction of duplicate entries made by Audit Bee. I have created a categories which collects the websites which were edited upon import more than once. Some names should be on our watchlist to track the edits. This category should be cleared before resuming the imports. Cheers --[[kgh]] (talk) 12:07, 27 January 2013 (UTC)

Excellent. I cleaned a number of them up this morning as well. Many of them seem to be changing URLs that the site owners are not managing correctly. I would like to just get this whole file in and then take the time to clean once the dust has settled. I think we'll have to do cleanup multiple times otherwise (like has happened with Wikinews a few times during this import.). I pulled out the stops and am pushing three imports in parallel with no delays. I'll have the entire file processed in the next couple of hours. (Interestingly enough, running 3 import processes in parallel isn't terrible on machine load since there is so much network waiting on requests. This is good news for parallelizing Bumble Bee, I bet I can run 3, 4 or maybe more in parallel.) Thingles (talk) 14:04, 27 January 2013 (UTC)
Yes you may proceed. There are names which will always be edited mulitple time. Perhaps we wait for them until everything moved in. This is the farm type of wikis. I will put them on my watchlist to be able to identifiy them. For all the others there is probably not a very big chance that they get edited again. Doing this on one day would be cool, because it is easier for me to identify the mulitple edits in the recent changes list. --[[kgh]] (talk) 14:16, 27 January 2013 (UTC)
Related, I'm thinking I'm going to move the "not validated", 'not active" flags into MediaWiki categories as well, and then put that all under Category:Site maintenance. Thingles (talk) 14:04, 27 January 2013 (UTC)
+ from me. I wanted to do this yesterday but it was too late for me at night. --[[kgh]] (talk) 14:16, 27 January 2013 (UTC)
Lastly, we need to put a "call to action" on the front of the site asking if people wish to do operator work, what they need to know and how to request it. :-) Thingles (talk)
As soon as we have worked on this lot it is time to announce this to the MW community too. Does not make sense earlier I guess. --[[kgh]] (talk) 14:16, 27 January 2013 (UTC)

Is defunct

Hmm, I am having second thought here. I think it would be better to be able to additionally mark a wiki as defunct in contrast to just deactivating it. Knowledge Base may serve as an example. What do you think? --[[kgh]] (talk) 17:15, 27 January 2013 (UTC)

Yeah, I have been thinking the same thing. How to properly identify the status of sites in various conditions. I'm going to think it out on Project:Website status flags, feel free to join the fun. Thingles (talk) 19:16, 27 January 2013 (UTC)
Nice page. I will add my thoughts tomorrow. --[[kgh]] (talk) 20:22, 27 January 2013 (UTC)

WikiApiary:Unvalidated wikis

Hey, what happened to this working list? I was just thinking of adding some randomness into my work since every wiki deserves a chance to be among the first ones to be tracked here next. This was also good as an aggregation of that is around. :( --[[kgh]] (talk) 20:19, 27 January 2013 (UTC)

Sorry. I deleted it when I made Category:Website not validated. I restored it and even added a second random table for ya. :-) Thingles (talk) 04:18, 28 January 2013 (UTC)
Admittedly there is a lot of congruence but this working list is really nice. Thank you --[[kgh]] (talk) 12:02, 28 January 2013 (UTC)

Extensions data for bulk imports

Hello, I don't understand: are you going to feed the new imports with extension data only after they're manually validated? I'd think at least a single run of such "stable" information could/should be done. Or maybe I'm not seeing the data in lists for some other reason. --Nemo 21:44, 27 January 2013 (UTC)

The extension data are indeed only collected after manual validation. You wouldn't believe how often the data for extensions change. Validation is also good for increasing the quality of the entries. --[[kgh]] (talk) 21:52, 27 January 2013 (UTC)
Yes but it won't improve data about extensions significantly (duplicate counting is not that big an issue for them, for instance). --Nemo 21:57, 27 January 2013 (UTC)

Category:Multiple website

I just tagged all pages in question as far as I could go since RecentChanges allows only 5000 changes and has no offset. Thus I had to stop at Openmoko (12:41). There may well be other pages too but currently I do not really know how to get hold of them. :| --[[kgh]] (talk) 14:45, 28 January 2013 (UTC)

It's surprising how many wikis are not managing their domain names and redirects right. :-( Thingles (talk) 15:01, 28 January 2013 (UTC)
I told you there were many in that list. ^^' I removed many by hand on the wikiteam lists, but there were still plenty which we noticed only after the wikistats import. --Nemo 18:53, 28 January 2013 (UTC)
The new installer introduced with 1.17 will hopefully help reducing misconfiguration. --[[kgh]] (talk) 19:00, 28 January 2013 (UTC)
Probably not at all, those are mainly apache/webserver misconfigurations. --Nemo 21:27, 28 January 2013 (UTC)