WikiApiary talk:Operations/2013/October

New Property:Has IP address
I had some emails with User:MarkAHershberger today about tracking wiki farms and he was wondering if WikiApiary could possibly get the IP address of wikis to match them up, even when farms aren't necessarily identified. I just added a little hack into User:Bumble Bee to do just that and they are now populating. Within 12 hours all will be picked up. Note that this is using the multiproperty capability in WikiApiary, so if a wiki uses multiple addresses over time it will pick them all up (this is currently used for DB versions). Screenshot of the data in the ApiaryDB:



Fun to watch this fill in. See if it tells us much. Also, jhe I wish Semantic MediaWiki had an IP address data type! :-)

🐝 thingles (talk) 01:41, 2 October 2013 (UTC)

''PS: I found a bug that showed up in this diff and have fixed it. FreeWiki/General will get fixed on the next update.''
 * nice... now to make use of this. -- ☠ MarkAHershberger ☢ (talk) ☣ 02:10, 2 October 2013 (UTC)

New Property:Has reverse lookup
Following up on Property:Has IP address it hit me that it might be really useful to do a reverse hostname lookup on the IP. Turned out to be really easy to add this as well, so that is now in. This is particularly useful since the reverse lookup tends to identify clearly the hosting provider. This is also being managed as a multiproperty.



🐝 thingles (talk) 02:23, 2 October 2013 (UTC)

PS: Mark, note Dreamhost listed in the graphic above.


 * Note this page User:Thingles/Hosting providers. 🐝 thingles (talk) 02:40, 2 October 2013 (UTC)
 * Probably noted elsewhere, but sorting by version number leaves 1.9 at the opposite end of the list than 1.10 with 1.20 in the middle. -- ☠ MarkAHershberger ☢ (talk) ☣ 13:41, 2 October 2013 (UTC)
 * posted to wikitech-l -- ☠ MarkAHershberger ☢ (talk) ☣ 15:02, 2 October 2013 (UTC)

Whois integration
Late last night/early this morning I added a whois query to bumble bee to augment the IP and Reverse hostname lookups. This now queries whois and adds the organization that owns the network block being used. See Property:Has netblock organization and Property:Has netblock organization handle. This will almost certainly allow groupings of wikis into logical units (all wikis at a University, all wikis hosted by Dreamhost, etc). The biggest limitation is that the Python library I used only queries ARIN, so addresses that are registered with one of the four other regional Internet registries bodies won't have detailed info yet. This is why you will see a lot of entries for "RIPE Network Coordination Centre". That is another RIR that I should then query to get the right answer. I'm open to code that would query them. ARIN will give good data for US and Canada. There is a really good Stack Overflow answer on Python libraries to do whois. Unfortunately they either only query ARIN, or don't allow IP lookups. :-( 🐝 thingles (talk) 13:18, 2 October 2013 (UTC)

New network fields
This diff does a good job of showing the new information coming into WikiApiary. 🐝 thingles (talk) 13:24, 2 October 2013 (UTC)

New property delay
Note that if you look for these new properties, they won't be populated on many wiki's entries until Bumble_Bee gets to them, so have a little patience. -- ☠ MarkAHershberger ☢ (talk) ☣ 15:20, 2 October 2013 (UTC)