User talk:Kghbln/Archive 01

Hello
Thanks for signing up and registering a website. I'm still actively building out WikiApiary but you will shortly see your statistics start to show up. Have fun and hope you find the Apiary useful! Thingles (talk) 20:13, December 28, 2012 (CST)


 * Heiya Jamie, I just saw that you added this site to the Community Wiki and thought I give it a try. A great idea and I am looking forward to how it develops. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 20:15, December 28, 2012 (CST)

Website Summaries
I saw you added summary/description text to some of the wikis you added. I've been thinking I should add that as a property for the website form. Seeing you add it makes me think that I definitely should! :-) BTW, thank you for adding more wikis. That is awesome! Thingles (talk) 07:41, December 29, 2012 (CST)


 * I really like this wiki. The idea of providing a brief description is from the smw community wiki. Having an extra field for this, including a related property is a good idea. In return you could remove the free text field. I am not so sure about the tags I should use and this is why I have not done is so far. Probably autocompletion on property values is the way to go here. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 07:52, December 29, 2012 (CST)


 * I've added the field to the form, added autocompletion and removed the free text area as you suggested (diff). Something is amiss with my autocompletion though. I've noticed this on another wiki of mine, need to debug what is happening there. Thingles (talk) 08:01, December 29, 2012 (CST)


 * Great! Looks good to me - approved! :) Why don't you just use "values from property" instead of "remote autocompletion"? Cheers --&#91;&#91;kgh&#93;&#93; (talk) 08:07, December 29, 2012 (CST)

Indexes on Wikis
I'm planning on adding a number of calculated indexes to WikiApiary shortly. Calculations using the existing statistics that would be interesting. One example I've been considering is an activity index that would be (edits in last 28 days/active users). Another would be admin index (active users/admins). Any suggestions you have on indexes would be great. Thingles (talk) 08:07, December 29, 2012 (CST)


 * You will see I made a Template:Website indexes that is now showing four calculated indexes for all wikis. I also added a table of "Most Active Wikis" to the Statistics page. Pretty fun! Thingles (talk) 08:24, December 31, 2012 (CST)


 * Yay, we two beat Wikidata :) Its getting better here by the day! --&#91;&#91;kgh&#93;&#93; (talk) 08:28, December 31, 2012 (CST)

Editor
I created a new Editors group and have added you to it so that I don't need to patrol your edits. :-) And also so that at some point Forms and Templates can be restricted but you can still edit. Any suggestions on other rights or groups that should exist would be very welcome. Thingles (talk) 06:32, December 30, 2012 (CST)


 * Great, thank you. :) One thing that comes to my mind immediately is . I will provide other suggestions along my way here. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 06:36, December 30, 2012 (CST)


 * Added! Thingles (talk) 06:40, December 30, 2012 (CST)

Changing subobjects
Just a heads up on a big change I'm making right now. Right now the bot writes extension and general information to /Extensions and /General subpages of a given website. The subobjects are then attached to that subpage with a linking Has website property. I'm changing this after doing some tests. I'm going to still write the subpages, but the data will be in  blocks. Then I will transclude the subpages into the website page.

This will have two wins. First, queries will be easier to create since all data will be in the website object or subobjects directly connected to it. Also, since a transclusion is in place when the bot changes the subpage it will trigger MediaWiki to update the page with the transclusion and get new data. This does mean some queries will have to be modified. Thingles (talk) 14:55, December 30, 2012 (CST)


 * I am touching wood. Support from me. :) Modifying the queries should not be the problem then. Cheers --&#91;&#91;kgh&#93;&#93; (talk) 15:10, December 30, 2012 (CST)


 * Change is made and should be propagated through. When you look at the properties for a website you'll see all the data now there. Thingles (talk) 15:19, December 30, 2012 (CST)

Header tabs
Funny! I was thinking that adding header tabs to the website template would be a good thing! See you just did that! Sweet. I'll pull the graphs out of the tabs though. I'm signing off for a bit. Youngest is done with nap and we're hanging out. Thingles (talk) 15:22, December 30, 2012 (CST)


 * I guess this was natural thing to do. :) Admittedly, I have to do something else, too. Enjoy your family time now! --&#91;&#91;kgh&#93;&#93; (talk) 15:26, December 30, 2012 (CST)

Sync with Semantic MediaWiki Community Wiki
I was looking briefly at the SMW Community Wiki on Referata and I think it would be pretty simple for me to write a bot that "synced" the websites registered there over to WikiApiary. Do you think this would be a good idea? Bad idea? Inappropriate? Cool? What do you think? In about 15 minutes I already have the query for pulling all the public, active sites from that wiki. Now I just need to make sure to not duplicate and attempt to discover the API endpoint my retrieving the base page and looking at the meta headers. Thoughts? Thingles (talk) 22:31, January 4, 2013 (CST)


 * A natural thing to think about this. Still I would not do it without asking Yaron about his thoughts, since he basically build up and fostered this directory. Otherwise it would appear like a bear taking a yummy honeypot. To some more or less large extent this wiki would supersede the community wiki after this was done. --&#91;&#91;kgh&#93;&#93; (talk) 05:13, January 5, 2013 (CST)


 * Thanks for the feedback. I agree, and I need to focus on getting the collector right before a bunch of hundreds of additional sites are added. Thanks! Thingles (talk) 14:59, January 6, 2013 (CST)


 * Once I get the rewrite of Bumble Bee (below) done and am actually honoring the Check every property I plan on publicly announcing this wiki on the SMW users mailing list. As it is, very few people know it exists. Thingles (talk) 15:05, January 6, 2013 (CST)


 * Still I would approach Yaron before announcing this wiki publicly since some people might see the similarity of both wikis and will ask about it. I am sure he will like this wiki anyway. --&#91;&#91;kgh&#93;&#93; (talk) 16:28, January 6, 2013 (CST)


 * Good idea. I don't know Yaron aside form some posts on the mailing list but when I get this bot stuff sorted I'll drop him a note, and CC you. Thingles (talk) 17:34, January 6, 2013 (CST)


 * Perfect! Thank you for doing this. --&#91;&#91;kgh&#93;&#93; (talk) 12:16, January 12, 2013 (CST)


 * Nice response from Yaron! Thingles (talk) 09:58, January 13, 2013 (CST)


 * Indeed. He is also a big fan of good and innovative ideas ... --&#91;&#91;kgh&#93;&#93; (talk) 10:05, January 13, 2013 (CST)

Bumble Bee on Github
Just FYI, I've started working on a rewrite of the three hacked together Python bots that collect information into one well written bot. If you want to see it there is a WikiApiary repo on Github.

Some highlights:


 * 1) When Bumble Bee cannot talk to a website (it's not responding, returns bad info) I plan on having it write a templated log message to the Talk page for that website. That make sense to you? Would there be a better place for it?
 * 2) Right now Bumble Bee isn't honoring the Check every setting. It pulls every 15 minutes regardless. This one will, but I'm going to move that from Check every hours to every minutes so that some sites (Wikipedia) can have 15 minute collection periods.
 * 3) I'm going to consolidate the three separate scripts that get stats, extensions and general info into this one.
 * 4) I had originally planning on pulling in the list of namespaces. You see that in the websites form. I'm reconsidering that. I'm not sure it adds any value unless I did a page count by namespace. That would be a bit harder than I want to do now, so just a list of Namespaces seems less useful.
 * 5) I'm totally excited about the Property:Has bot segment and running multiple Bumble Bee's at once. I've already got that in the rewrite. I could easily run 4 of these at a time and keep stats collecting very fast without having to write threading code.

Anyway, that is why you see a little less of edits in wikispace from me right now.

Thingles (talk) 15:04, January 6, 2013 (CST)


 * Thank you for keeping me in the loop. After you added the new property I found my way to GitHub. By the way, I am following you now. :)


 * …and I you! :-) Thingles (talk) 17:35, January 6, 2013 (CST)


 * ad 1) That's a good idea. This info belongs to the page since this is a page the admin of the wiki will most likely watch. Still adding semantics to the template for it to be also queried on a central maintenance page is favourable as well as the possibility to update this info within the template in a sense of "I worked on the problem successfully. Done."
 * ad 2) I was already wondering about these checks since my impression from looking at the graphs was that it is doing them much more frequently. My weblog is fully of bee activity too. :) So Bumble Bee is presently as busy as a bee can get. :D Going to minutes is good. Besides, since Bumble Bee is doing these checks on a regular basis anyway an information about the uptime of the respective wiki could probably easily be added to the website's page. I guess this will be very interesting at least for the admins too.


 * If I tried to add availability monitoring as well I would probably put that in a separate bot by itself and have that run every 5-minutes. Not in my plans now, but who knows. Thingles (talk) 17:39, January 6, 2013 (CST)


 * I see. Fair enough. Still an admin fave. :) --&#91;&#91;kgh&#93;&#93; (talk) 13:58, January 7, 2013 (CST)


 * ad 3) When doing this it would be great to have the new structure of the template in mind, i.e. to split up the General information so it may be easily allocated to the new sections. However, as I got to know you up till now, you have already concrete plans to cater for this. :)


 * I plan on leaving the templates as is, just having them all generated in the same Python as opposed to three separate scripts. Thingles (talk) 17:39, January 6, 2013 (CST)


 * Ah, we will have to see how to allocate the infos from the template. Will involve querying. --&#91;&#91;kgh&#93;&#93; (talk) 13:58, January 7, 2013 (CST)


 * ad 4) I was wondering about this one too. However, I did not put much thought into it since I regarded this as a less important information. It is indeed less helpful having the purpose of this website in mind.


 * Unfortunately I don't see any method in the Mediawiki API that allows me to ask for a count of pages in a given namespace. It looks like I would have to ask the wiki to give me the list of page names and then count them, which I think is too expensive. I can envision some crazy future where there is a WikiApiary Extension that could optionally be installed and could collect this type of information locally and present it in a nice API. That will go on the Roadmap. Thingles (talk) 17:39, January 6, 2013 (CST)


 * Probably only evolved wikis use more than the ones provided out of the box. In case you have extra namespaces you could set up  to have their pages included in regular statistics. Having an extra extension to do this will be an option, too. Not on the front burner tough. --&#91;&#91;kgh&#93;&#93; (talk) 13:58, January 7, 2013 (CST)


 * ad 5) A cool idea which will work out great!
 * Cheers --&#91;&#91;kgh&#93;&#93; (talk) 17:01, January 6, 2013 (CST)
 * PS I thought you might have another life with wife and kiddies and a job. No worries about your activity. :)
 * If there were just three of me it would be so much easier to get stuff done! :-) Thingles (talk) 17:39, January 6, 2013 (CST)

Wikipedia Jobs Graph
Check out the jobs graph for Wikipedia (en). Interesting build up. At some point it would be interesting to trigger a notification to admins when jobs are not clearing fast enough. Thingles (talk) 09:59, January 7, 2013 (CST)


 * :) Probably there are some people constantly fiddling around with templates such as taxoboxes etc. at the moment. + from me. --&#91;&#91;kgh&#93;&#93; (talk) 14:00, January 7, 2013 (CST)

Reconsidering Farms
I've been reconsidering the issue of wiki farms that you brought up before. I'm thinking that my idea of tagging isn't really enough for farms/hosters/platforms. Tags are great for a bunch of things, but the farms themselves end up having data. I'm thinking that farms should be a specific (single) property of websites, and that those should populate "farm pages" in another namespace (very much like the extension namespace). Those pages will then be their own objects and hold aggregated data for the form itself. I think that would be the place where a farm administrator would go to see status across their platform.

I'm not eager to attempt to import 30,000 Wikia wikis, but I am eager to see aggregated performance across my 14 or so wikis in my garden. And I can definitely see some smaller platforms with hundreds of wikis using WikiApiary actively.

What do you think? And if you agree, what would the namespace and property be called? I see one property of a farm being it's type or classification which would be autodetermined based on the number of wikis in the farm ( < 20 = garden, < 100 = ???, < 1000 = ???). Thingles (talk) 06:57, January 8, 2013 (CST)


 * I think that this is a good idea as you may already have expected. :) One thing that could be avoided is the statistical imbalance I already mentioned, the other is the specific aggregation you just mentioned. I would call the namespace "Wiki farm" since this seems to be the established and most commonly used term for it. I would however not do a named classification therein based on the number of instances within it as you proposed. There are just not enough of them around and the naming would be arbitrary with minimum additional benefit. In case you insist on this you could just call it "small wiki farm" ( > 10), "medium wiki farm" ( > 100), "large wiki farm" ( > 1000). The minimum number of wikis to qualify for the "Wiki farm" namespace should be 10 I guess. --&#91;&#91;kgh&#93;&#93; (talk) 09:00, January 8, 2013 (CST)


 * Excellent. I'm thinking just "Farm" and "Farm talk" for the namespaces, and Property:Has farm. I planned on just making these automatically created pages, similar to extensions. This would not allow for a minimum instance count before creating though. It does seem to me that a property of a farm should be the website count in it, regardless of whether that is named. It would also think that a farm type would make sense (commercial, non-profit, hobby). Thoughts on other properties of farms? I'll wire this all up in the next day or two. Fun! :-) Thingles (talk) 09:14, January 8, 2013 (CST)


 * Yeah, go for it! "Farm" and "Has farm" is ok, also the properties about size and type. Perhaps a property about establishment would be nice to have, but I am not so sure about it. --&#91;&#91;kgh&#93;&#93; (talk) 09:34, January 8, 2013 (CST)


 * By establishment do you mean the organization (if any) that runs the farm? Thingles (talk) 09:57, January 8, 2013 (CST)


 * Yep, Wiki was established in 2004 I guess, your farm came into existence in ... --&#91;&#91;kgh&#93;&#93; (talk) 09:59, January 8, 2013 (CST)


 * Farms now exist! Farm:Thingelstad.com I decided to not automatically create the farm pages. Thingles (talk) 20:02, January 8, 2013 (CST)


 * Not to do it automatically will not be a problem since there are only two hands full of farms around anyway. A great thing to have them. --&#91;&#91;kgh&#93;&#93; (talk) 14:19, January 9, 2013 (CST)


 * I fluffed up the templates and forms for farms. :) Cheers --&#91;&#91;kgh&#93;&#93; (talk) 16:29, January 10, 2013 (CST)


 * Thanks — looks awesome! Thingles (talk) 21:57, January 11, 2013 (CST)


 * And thank you for rectifying the shortcomings of my changes! --&#91;&#91;kgh&#93;&#93; (talk) 09:30, January 12, 2013 (CST)

✅

Bulk Loading
Check out Bulk import websites. Clearly the quickest way to add a bunch of sites, if so desired. I used it to import 13 websites tonight as a test, just pulling one of the groups from S23 WikiStats. Worked easy. Thingles (talk) 21:09, January 8, 2013 (CST)


 * That's definitely the way to go. The cool thing is, that Data Transfer recognises existing pages and does not import if instructed. So, no typos in titles please. :) --&#91;&#91;kgh&#93;&#93; (talk) 14:18, January 9, 2013 (CST)

Redid Main Page
Took a first attempt at a total revamp of the Main Page. Decided to try using the article count as a grouping mechanism. Step in the right direction... Thingles (talk) 21:57, January 11, 2013 (CST)


 * Definitely a good change. :) I just adjusted the format of the general stats section. Probaby some of it may go into a separate class. I moved the bot stats out since this is not interesting at all for regular visitor. Also they are unrelated to sites and farms anyway. Please move it in again in case to think it should be there. --&#91;&#91;kgh&#93;&#93; (talk) 09:33, January 12, 2013 (CST)

Lot's of sites!
Wow — you are adding sites like crazy! That's awesome! Thanks! Thingles (talk) 09:22, January 13, 2013 (CST)


 * Yeah, but for the time being only sites of which I believe that they are not properly listed elsewhere, e.g. Biowikifarm wikis. The rest is for Bumble Bee. :) Cheers --&#91;&#91;kgh&#93;&#93; (talk) 09:25, January 13, 2013 (CST)


 * I added the Plantnet farm under the biowiki farm. I like the ability to have farms inside of farms. :-) I actually think for large imports I'm gonna just do a CSV bulk import rather than a bot. Main area to use a bot would be to maintain synchronization, but if it's just a one time load a CSV will be easier. Thingles (talk) 09:57, January 13, 2013 (CST)


 * Hey, that's what I kept for tomorrow to do. :) Ah, I remember that we already talked about the CSV import. You are right, this is what I meant. A hundred a day from the MediaWiki section (unaffiliated wikis)? --&#91;&#91;kgh&#93;&#93; (talk) 10:09, January 13, 2013 (CST)