New Software Announcement - SIXTH SENSE

[LVGamb00ler]

I agree with TeddyFBI regarding aggregation across limits. I was in fact surprised to see that SS doesn't do that.

I hope that the modifications to implement Teddy's recommendation don't invalidate the PTagg DB that just took about 40+ hours to run on my 3.4M hand PT DB. I'm sure glad that PTagg is written to allow it to be stopped/re-started. I ran it about 10 hours at a time, then stopped/vacuumed/restarted.

Also if you have any documentation on the PTagg schema, I would like a chance to read it. I do some direct manipulation of the PT DB's and probably need to do the same for the PTagg DB to keep them consistent with the PT copy.

Good luck all,
LVGamb00ler

[teddyFBI]

[ QUOTE ]
Thanks Teddy, that makes a lot of sense.

I think I'll add an option to SS (which will be used in PTAgg), to query the players only based on # of players (using my defaults) and limit vs. no-limit (so regardless of stakes). This will be the default option. I'll add another option to just do that game-type/stakes. And finally an option to pull from all game-types/stakes.

I don't want to change the # of players queried if I can help it. Its some work, and doesn't have much payoff.

I think that should get SS 98% there.

-Ben

[/ QUOTE ]

All of that sounds reasonable to me, and should look good.
Rigid 5-6 and 8-10 filters seem reasonable to me. It still won't fulfill your goal of having PTAGG/SS reflect EXACTLY what PAH pulls up (b/c people use all sorts of different PAH filters; mine, for example, are 4-6 and 7-10).

BUT, there is one issue left open, which shouldn't be too hard to solve, but you should be ready with a good solution.

So it's clear that when PTAGG comes across a table with 8-10 people sitting at it, it will say to itself:
"OK, let's look up each of these players' stats from hands that ALSO had 8-10 players being dealt cards."

Same goes for the 5-6 range. No problems there either.

But you'll have to decide how PTAGG/SS is going to treat 2-4 person games and 7-person games. It's fine to define SH as 5-6 and "full" as 8-10, but you have to decide what PTAGG is going to do when it sees a 7-person game going, for example. My personal recommendation? Treat it like a SH game...note that this DOESN'T mean you have to expand your filter from 5-6 to 5-7. All it means is that you have to be sure to tell PTAGG: "Hey, when you come across a 7-person game, I want you to display stats using your 5-6 filter.

Same goes for 2, 3, and 4-handed play. I definitley think you should keep the actual query that PTAGG does restricted to 5-to-6-person hands...why? b/c I think that 6-max games are sor of the norm, as discussed on internet forums, etc., and everyone is familiar with 6-max stats, and what a regular range is, etc. When I sit in a 4-handed game, for example, and I see an opponent's VPIP/PFR is 45/30 for 5-6-handed games, I know that means he's quite a bit LAGgy. But frankly, most people don't know what the "norm" is for a 3-handed or 4-handed game, because those aren't discussed as much as 6-handed games are. So it's just in line with keeping with the norm; hope the preceding made sense.

Cliff's notes is to just tell PTAGG to populate the query of a 7-handed game with stats from your 8-10 person filter, and to populate the query of 2-to-4 handed games with stats from your 5-6 filter.

Also, do you have any idea what is causing the VERSION_INFO error in PT that I referenced earlier that pops up when I try to open the ptagg database just like any other DB? That would make troubleshooting this whole thing a lot easier, b/c I could just go check what the ptagg DB has as the aggregated stats...and see if it's correctly matching up w/ the stats from the indiv DBs that went into creating it.

[SixthSense]

[ QUOTE ]
I agree with TeddyFBI regarding aggregation across limits. I was in fact surprised to see that SS doesn't do that.

I hope that the modifications to implement Teddy's recommendation don't invalidate the PTagg DB that just took about 40+ hours to run on my 3.4M hand PT DB. I'm sure glad that PTagg is written to allow it to be stopped/re-started. I ran it about 10 hours at a time, then stopped/vacuumed/restarted.

Also if you have any documentation on the PTagg schema, I would like a chance to read it. I do some direct manipulation of the PT DB's and probably need to do the same for the PTagg DB to keep them consistent with the PT copy.

Good luck all,
LVGamb00ler

[/ QUOTE ]

Don't worry, you will not have to run the aggregator again. Any changes that are made will only be to the query that SS makes to the ptagg database. That doesn't require a re-import.

Its good to see that the import time is averaging about ~1 hour per 100k hands, even on very large databases. I know this is a long time, but I think the query speed ups will make it worth it. And even more so if PAHud decides to support ptagg as a datasource.

There's no documentation on the ptagg schema. But you can use PGAdmin III (which comes with postgres) to open up the database and look at the tables and fields.

What types of manipulation do you do? Remember that ptagg is a "summary-only" database. It has no detail records. So any updates or deletes you do on a PT database might not make sense on ptagg.

-Ben

[SixthSense]

[ QUOTE ]
Cliff's notes is to just tell PTAGG to populate the query of a 7-handed game with stats from your 8-10 person filter, and to populate the query of 2-to-4 handed games with stats from your 5-6 filter.

Also, do you have any idea what is causing the VERSION_INFO error in PT that I referenced earlier that pops up when I try to open the ptagg database just like any other DB? That would make troubleshooting this whole thing a lot easier, b/c I could just go check what the ptagg DB has as the aggregated stats...and see if it's correctly matching up w/ the stats from the indiv DBs that went into creating it.

[/ QUOTE ]

Yes, the way I'm doing it now matches your "cliff notes" version. I think this would only be a problem for players that specialize in short-handed games. I'm not sure how many that would be?

As for the VERSION_INFO error, the reason is because ptagg is NOT a real PokerTracker database. It has PT data in it, but in a completely different format, with different tables and fields. A query that works on a PT database will not even run on ptagg, it needs its own special queries.

I could add a "lookup player" menu to PTAgg if that would be helpful. I could have it show the summary stats, as well as the "detail" stats (in this case detail just means the summary stats broken down by year, month, game_type, # of players).

-Ben

[goodguy_1]

I am using the PT Aggegator and have choosen to aggregate 2 databases-my main playing database(about 800K hands) and a dataming database. The datamining database has 2.23 Million hands and PT Aggregator is telling me at the current pace-it will take 5 days to complete the importing of all hands-is this right? Wow -that's slow! anyway to make the process faster.

This app. is awesome btw!

[SixthSense]

[ QUOTE ]
I am using the PT Aggegator and have choosen to aggregate 2 databases-my main playing database(about 800K hands) and a dataming database. The datamining database has 2.23 Million hands and PT Aggregator is telling me at the current pace-it will take 5 days to complete the importing of all hands-is this right? Wow -that's slow! anyway to make the process faster.

This app. is awesome btw!

[/ QUOTE ]

The Est. time remaining is almost useless as predicting how long it will take. If your big database is first, it will overestimate by a lot, if its last it will underestimate by a lot.

The only way to speed it up is to shut down all other apps, that will give PT Aggregator the resources it needs. If you leave PartyPoker running, it will slow down PT Aggregator a great deal.

It is advisable to stop & restart PT Aggregator (don't worry, it will start where it left off). This because there is still a known memory leak in the import process. It will be fixed in a future version.

-Ben

[SixthSense]

Build 40 of Sixth Sense has been Posted:

v1.0.0.40 - 8/14/06

- Added: Added new option "Scoring Method" to Advanced Options (CTRL-O)
- "Match GameType, Any Stakes" (Default)
- "Match GameType, Match Stakes"
- "Any GameType, Any Stakes"
- GameType means Limit or No-Limit/Pot-Limit
- Stakes means size of blinds/buy-in
- Added: Info dialog after Auto detect databases that says how many
new and existing database(s) it found
- Added: New dialog in Config Databases Dlg to tell user that they need to
locate their PokerTracker Directory (to find any PT Access db's)
- Added: User's Table Sorting preference is saved between sessions
- Added: User's Column Ordering preferences (Table & Players) are now saved
between sessions
- Added: A warning dialog if the SS server returns 0 tables
- Added: A warning dialog if SS can't score any players using your database(s)
- Added: Buddy Dialog now has a field for "Last Seen"
- Changed: Scoring dialog now reports how many players have stats (%)
- Changed: Players with no hands will now be flushed from the cache after 5 minutes
- This should make it easier to see if new players are being imported into PTAgg
- Changed: Buddy name column in Buddy list is now wider

-Ben

[teddyFBI]

I will test this tonight.

Suggested quick fixes for the next update:

1) You were going to remove one of Options...Weighting for Table Averages and MENU...Weighting for Table Averages. You said they did the same thing, so it's hard to figure out which one you have made obsolete

2) You should call the Menu...Advanced submenu Menu...Stat Cache Options or something...just avoids a bit of confusion when you say go to "Advanced"...b/c you also have Menu...Options...Advanced Options.

3) I think you need to beef up your website/tutorial; e.g. In Advanced Options... I have no idea what each of those line items does...well i have some idea but your average user will not, and will prob not know what he can and cannot change, and what it will do.

4) I also suggest you streamline your dropdown menu's by removing any option that you can get to directly from your main menu bar. e.g. you can now access "COnfig DBs" and "Games" from the main menu bar...i'd suggest removing those, then, from the dropdown menu...KISS

5) Oh, I also suggest a "Stop Search" button on the main menu bar, or just above the status bar. Sometimes I want to just stop a search that I see is taking a while (pre-PTAGG of course), and i think everyone feels more comfortable if there's an explicit 'stop search' button to press...i don't want to have to wait for a search to complete if i want to change the search parameters or something; i'd like to be able to kill the search w/o stopping the program. again: PTAGG might make this request obsolete.

6) Minor annoyance: when i drag a column to place it in a different location, i don't want SS to think i'm ALSO asking it to sort by that column...i only want to move it left or right...so tell SS to ONLY sort when a user clicks a column header WITHOUT moving it somewhere...if i drag and drop it to the right or left, it shouldn't sort.

7) Oh, add a Waiting List column in the Table List. (if i see 7 juicy tables i want to open, i want to know which has the longest/shortest WL's

8) You still need a user option for "Min # of players"; e.g. i would wager that most (not all, but most) of your users want to exclude from their searches tables with 3 or fewer players at them...let that be a user-defined number in your Advanecd Options line-item menu.

First impressions, BTW: this is already night and day better than the last version. HUGE improvement...but I've been trying to check it against my laptop (non-PTAGG), but I can't b/c EVERY time I open try to run a search on my laptop, I get:

Error: server closed the connection unexpectedly. This prob means that the server terminated abnormally before or while processing the request.

What does that mean?
Seems worrisome, as it's not letting me run any searches...

EDIT#2: I think I figured out why that error was occuring, and it suggests a bug that should be fixed: Last night I had SS drawing from 5 DBs...Today I realized one was obsolete, so I deleted it from PT (not just from the maintain database name menu, but i used the Delete DB from the postgres DB functions menu...so it was completely gone). HOWEVER, when i kept running SS's automatic DB configuration utility, it kept finding that DB anyway, even though it was permanently deleted...it seems to think that if a user once had a DB selected in that configuration utility, than it doesn't need to bother checking to see whether it still actually exists...so it kept showing up as a "found DB", and stayed checked...until I manually unchecked it. Now the error doesn't happen anymore...i think it was b/c SS thought that I was asking it to get stats from a DB that did not exist (but one which its auto-detection utility could not figure out didn't exist anymore...hope that made sense)

[teddyFBI]

Something still seems to be funky w/ the stats SS / PTAGG displays for v shorthanded tables, e.g. 2 or 3 ppl.

I opened up a 100/200 table with 2 players thinking they were both huge fish w/ 60+ VPIPs, only to have my PAHUD tell me that i have 10K hands on each, and their VPIPs are around 30. (SS/PTAGG told me i only had 100-200 hands on each).

So something's wrong. In fact, one of the two players is actually at one of my 6-max 30/60 tables. So I checked the SS/PTAGG query again, and sure enough SS/PTAGG found him at a 30/60 table with 6 players and (correctly) showed his stats to be around 10,000 hands w/ VPIP of 32, and IN THE SAME SEARCH showed him at a 100/200 table with only 2 people at it with 53 hands, and a VPIP of 66.

What gives?
If, as I thought we had agreed on, you were going to have SS/PTAGG treat 2 and 6 person table exactly the same (e.g. query stats for 5-6 person hands), one player's stats should be exactly the same at a 2 and a 6-person table.

[SixthSense]

[ QUOTE ]
Something still seems to be funky w/ the stats SS / PTAGG displays for v shorthanded tables, e.g. 2 or 3 ppl.

I opened up a 100/200 table with 2 players thinking they were both huge fish w/ 60+ VPIPs, only to have my PAHUD tell me that i have 10K hands on each, and their VPIPs are around 30. (SS/PTAGG told me i only had 100-200 hands on each).

So something's wrong. In fact, one of the two players is actually at one of my 6-max 30/60 tables. So I checked the SS/PTAGG query again, and sure enough SS/PTAGG found him at a 30/60 table with 6 players and (correctly) showed his stats to be around 10,000 hands w/ VPIP of 32, and IN THE SAME SEARCH showed him at a 100/200 table with only 2 people at it with 53 hands, and a VPIP of 66.

What gives?
If, as I thought we had agreed on, you were going to have SS/PTAGG treat 2 and 6 person table exactly the same (e.g. query stats for 5-6 person hands), one player's stats should be exactly the same at a 2 and a 6-person table.

[/ QUOTE ]

In the case where there are just 2 people at the table, SS is defaulting to the heads-up stats (I had forgot about this in my previous post). I could make this an option. However, I plan on implementing the "Min # Players" option, so I think this will make it a moot point to you.

And thanks for the suggestions in your previous message. I'm pretty sure I can implement those.

-Ben