Page 1 of 2
Posted: Sat May 01, 2010 9:13 pm
by TheCatt
I think I figured out how to write a tool that will capture the JSON going on in the browser, and then take the results and spit it into the database.
That means anything you can display (rankings, player locations, raids, etc) can be automatically processed into a DB just by loading it on your screen.
I've proven that I can capture and interpret the data, but I haven't worked on parsing and storing it yet.
Posted: Sat May 01, 2010 9:32 pm
by GORDON
Be careful. That sounds like you are capturing the data stream... could be bannable.
Also, YOU are the new tool.
Posted: Sat May 01, 2010 9:34 pm
by TheCatt
BOO YAH.
It worked. Kinda.
OK, I'm just storing the request and response stuff right now, But, the data contains unique IDs for almost everything (including reports), meaning that if you accidentally double-click on stuff, it doesn't matter, everything can still be uniquely identified to remove duplicate data.
Rock on.
I'd love to see this used to capture spy reports, etc.
Posted: Sat May 01, 2010 9:36 pm
by TheCatt
GORDON wrote:Be careful. That sounds like you are capturing the data stream... could be bannable.
Also, YOU are the new tool.
Usually the way these things work is that capturing a data stream is fine, but that any attempt to automatically send a data stream, tamper with a data stream, etc, is banned. So we cannot automate going through the top 500 players, but if we want to clicky-clicky through, quite frankly - there is no way for them to know about it, unless word gets out.
So... word cannot get out. And we can only tell alliancemates that we IRL trust or such.
Posted: Sat May 01, 2010 11:44 pm
by Cakedaddy
As far as the message board goes, I figured all the really sensative stuff would be discussed here, instead of the other message board. Double security. On the other message board, we could accidently give acess to the wrong person. Here, they aren't even registered. So, keep really sensative stuff here.
As of now, the Officer forums are protected such that only we (people who visit this site and our friends/family) have access. Eventually, I could see promoting other people to officer so they can participate in political/strategical discussions. So, even though Officer is just us for now, it could change in the future. So, all very sensative stuff should only be discussed here.
Posted: Sun May 02, 2010 9:39 am
by TheCatt
This morning I wrote the code that parses out the raid reports JSON text into objects. The next step would be to convert it to the database stuff. There are a few variables that I have no idea what they mean, but I can just ignore and store them for now.
Posted: Mon May 03, 2010 6:00 pm
by Cakedaddy
I know this would be HIGHLY bannable, but. . .
If you can read the incoming stream, are you also able to monitor the outgoing stream? And if so, could you then copy it? And if so, could you combine the two to create your own building minister?
Posted: Mon May 03, 2010 6:22 pm
by TheCatt
I'm 99% sure that yes, I could.
It's all JSON, and everything that is browser-based is forgeable one way or another. The particular tool I'm using is completely transparent. It uses an actual browser session, and just makes copies of the stream in each direction. It's basically a debugging tool. But it can change the streams as they go through.
That being said, it would take a fair amount of effort to do it, but I'm pretty sure it's doable.
Posted: Mon May 03, 2010 6:59 pm
by TheCatt
Here's the code to send an upgrade, and then the response from the server (which I assume is a success/failure flag):
Request:
{"session":"30bedeff-3076-4113-aadf-6116fdd2b33e","cityid":"35651875","buildingid":67850,"buildingType":4,"isPaid":true}
Response:
true
The above was for a Cottage, so that's buildingtype 4.
All that has to be done is know what your cityid, buildingid, and buildingtypes are. isPaid I assume is "do i have a bldg minister" Which is interesting that it'd be something that the client sends, and not just known on the server.
I'd have to do a request for a new bldg to see how it handles non-existing buildings.
You can also query your buildings, and get info like:
{"b":0,"cl":5,"i":4,"l":7,"ml":[{"l":5,"m":[{"p":135,"t":13,"v":0},{"p":200,"t":6,"v":0}]},{"l":6,"m":[{"p":146,"t":13,"v":0},{"p":220,"t":6,"v":0}]},{"l":4,"m":[{"p":125,"t":13,"v":0},{"p":180,"t":6,"v":0}]},{"l":7,"m":[{"p":158,"t":13,"v":0},{"p":240,"t":6,"v":0}]}],"mp":[{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25}],"mr":[{"b":0,"l":1,"m":15,"v":35}],"mrd":[],"p":[],"t":3314,"down":{"b":16384,"cl":5,"i":4,"l":5,"ml":[{"l":5,"m":[{"p":135,"t":13,"v":0},{"p":200,"t":6,"v":0}]},{"l":6,"m":[{"p":146,"t":13,"v":0},{"p":220,"t":6,"v":0}]},{"l":4,"m":[{"p":125,"t":13,"v":0},{"p":180,"t":6,"v":0}]},{"l":7,"m":[{"p":158,"t":13,"v":0},{"p":240,"t":6,"v":0}]}],"mp":[{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25},{"b":6,"m":49,"v":25}],"mr":[{"b":0,"l":1,"m":15,"v":35}],"mrd":[],"p":[],"t":909}}
Which I don't really know what that means, but could figure out.
Posted: Tue May 04, 2010 12:51 am
by Cakedaddy
Very cool. So, do that instead of work tomorrow!
I'm not even talking about a full blown minister. Even something that issued upgrades for you at a specified time would be cool. My in game queue is full, but should have open slots in an hour. So, in an hour, issue a cottage upgrade. 20 minutes after that, issue a quarry upgrade. Something like that.
One risk would be if they had to shut something down for maint, but still see request traffic coming in. Kind of a red flag there.
Posted: Tue May 25, 2010 8:24 am
by Cakedaddy
I see you are monitoring, or have access to player ID/history from the thread on your message board. Are you able to get alliance, coords of cities and city type (castled)? If you can get the data, I'll write the app that parses and maps it. Although, I'm sure you can do that MUCH quicker than I can. This way you don't have to do it all.
Posted: Tue May 25, 2010 9:27 am
by TheCatt
Here's where I'm at:
Each day, I startup an app that captures the JSON traffic. I then go through the player and alliance rankings (I click on each, then hold page down until they are done, takes about 5 minutes). I then have another app that I click 1 button and it parses all that info.
So player and alliance history is in a decent shape.
I've also written code that parses the map. So as you navigate the map, the above app will capture the information and record it in the DB. But it's kinda a pain to navigate the map each day.
I've avoided writing bots, since they are ban-worthy.
If you want, I can start posting information on the app, with links. It's not perfect, but decent enough.
All the information is then stored in a SQL Server database. I can provide an account that will allow you to work with it. I'm creating some pages on http://www.thecatt.net/lou, but have been pretty busy lately.
I have some sample data in the SQL DB for cities if you want to work an a mapping app.
Posted: Tue May 25, 2010 11:13 am
by TPRJones
TheCatt wrote:I then go through the player and alliance rankings (I click on each, then hold page down until they are done, takes about 5 minutes).
If you've switched to Google Chrome, check out an extension named iMacros for Chrome. You can probably automate all that clicking and scrolling instead of doing it by hand.
Posted: Tue May 25, 2010 12:55 pm
by TheCatt
iMacros exists for FireFox as well... I think it has issues with Flash apps, however. Although it might be able to do it by using X/Y coords instead of object references.
The bigger issue is that my app only captures IE traffic right now. I'm pretty sure it can do FireFox, I would just have to modify some FF settings.
Posted: Sat Jun 19, 2010 8:42 pm
by TheCatt
So the scouting report actually tells you EXACTLY how many bldgs the other player has, and what level every single one of them are. But the UI summarizes it into "total levels"
Interesting.
For instance, I scouted someone today and it showed Woodcutter's Huts at 119. However, the actual JSON showed:
{a:3,l:7,t:1}
{a:4,l:6,t:1}
{a:2,l:9,t:1}
{a:7,l:8,t:1}
Where a is the count, l is the level, and t is the type.
So 3x7 + 4x6 + 2x9 + 7x8 = 21 + 24 + 18 + 56 = 119 [type 1 = woodcutter's hut]
Posted: Mon Jul 05, 2010 7:22 pm
by TheCatt
I started trying to see what I could do about managing my cities today.
I figured out how to switch cities automatically.
Now I just need to figure out what the JSON means, how to display bldgs queues, how to add, etc, etc, etc, etc.
21 wide (wall to wall), 21 tall (wall to wall)
Posted: Wed Jul 07, 2010 1:03 pm
by TheCatt
I started this today. My goal is to create an alpha version that can successfully A) Pick up a list of cities, B) Navigate to a selected city, and C) Send the build moonstone command.
Posted: Wed Jul 07, 2010 3:33 pm
by TheCatt
Day 1: I created an application that will sniff the network traffic, created a few classes to help store the data, and was able to successfully sniff the session ID from a LoU session.
I also created a rudimentary UI that has debug information, and allows you to turn the sniffer on and off.
So next I need to add parsing of the Player Information.
Then parsing of the City information.
Then parsing of building information.
Then display logic/UI for city and bldg information.
Then command information for navigating to a city, and building a moon stone.
Posted: Wed Jul 07, 2010 3:44 pm
by Cakedaddy
Wireshark is a good sniffer as well. Full on network sniffer. Unless you are building on your sniffer for functionality. . . If you are using it just to learn the data stream, you might like wireshark because you can have it sort and it will group packets based on the session and stuff. So, it should put all lou related packets together, in order.
Posted: Wed Jul 07, 2010 3:53 pm
by TheCatt
I'm using Fiddler. It puts all of the request and response information together, so higher level than just doing packets.