New Software Announcement - SIXTH SENSE

[SixthSense]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
how often is it supposed to update? been half an hour now

[/ QUOTE ]

Do you mean Sixth Sense's Table grid? That only updates when you press the Refresh button.

-Ben

[/ QUOTE ]

When I posted that, your server wasnt refreshing properly and was still giving me table stats from an hour ago.

[/ QUOTE ]

Ahh, I see.

Yes, normally the data from our server should be < 5 minutes old. If its more than that there is a problem on our end.

Yesterday we had a router failure, so that's probably what you saw.

-Ben

[Dazarath]

I can't figure out where the "open multiple tables" function is? Could someone help me out?

[SixthSense]

[ QUOTE ]
I can't figure out where the "open multiple tables" function is? Could someone help me out?

[/ QUOTE ]

Its on the Table Grid's Popup-menu. You can right-click on the table grid, or click the "Table Menu" link that's in the Table grid's header on the left. Then choose "Open Multiple tables". The shortcut is CTRL-Z.

Note - Open Multiple Tables will use your default Open Table Method. Open Advanced Options (CTRL-O) to read about the different ways it can open your tables.

-Ben

[goodguy_1]

I hadnt compacted the ptagg database for a day or two and I noticed PTAgg was eating up 1.10 GB of RAM,after vacuuming twice in a row..now it using only about 200K of RAM-vaccum often if you are doing alot of importing.

Why is PTagg.exe so sensitive to having simultaenous apps running? I have combined aggregated db's now of ~3.50Million hands.

Even when all I run is ptagg.exe I still can only import ~25K hands per hour at best. With other programs up and running I can import ~6K-10K hnads per hour. I still want to import a few more large mining database but at the rate the imports are slowing down to-this may not be doable [img]/images/graemlins/frown.gif[/img]

I have 4GB of RAM too.

PTaggregator importing really seems to slow down as you add more hands to the ptagg database.

I backed the ptagg db to a backup harddrive-I dont want have to do this again!

Where is the bottleneck ?
Why are succesive importing jobs slowing down-is this a psql issue?

I backed up the ptagg db and it's is very small in size- only ~8MB for ~3.5Million aggregated hands.

Also I noticed changing the batch size from 100 to 500 or 1000 did not make imports faster.

[Dazarath]

One more question about subscriptions. I signed up for a full year subscription (still in the middle of the trial right now). Would it be possible to switch it to one of the shorter plans?

[SixthSense]

[ QUOTE ]
One more question about subscriptions. I signed up for a full year subscription (still in the middle of the trial right now). Would it be possible to switch it to one of the shorter plans?

[/ QUOTE ]

Yes, just go in and cancel it (which is OK because its still before the 15 days are up), then sign up for a different plan.

Note that as soon as you cancel the first one, your trial is immediately over, so you'll have to sign up for the new one right away.

-Ben

[SixthSense]

[ QUOTE ]
I hadnt compacted the ptagg database for a day or two and I noticed PTAgg was eating up 1.10 GB of RAM,after vacuuming twice in a row..now it using only about 200K of RAM-vaccum often if you are doing alot of importing.

Why is PTagg.exe so sensitive to having simultaenous apps running? I have combined aggregated db's now of ~3.50Million hands.

Even when all I run is ptagg.exe I still can only import ~25K hands per hour at best. With other programs up and running I can import ~6K-10K hnads per hour. I still want to import a few more large mining database but at the rate the imports are slowing down to-this may not be doable [img]/images/graemlins/frown.gif[/img]

I have 4GB of RAM too.

PTaggregator importing really seems to slow down as you add more hands to the ptagg database.

I backed the ptagg db to a backup harddrive-I dont want have to do this again!

Where is the bottleneck ?
Why are succesive importing jobs slowing down-is this a psql issue?

I backed up the ptagg db and it's is very small in size- only ~8MB for ~3.5Million aggregated hands.

Also I noticed changing the batch size from 100 to 500 or 1000 did not make imports faster.

[/ QUOTE ]

PTAgg has a known memory leak, so its best to shut down the app and restart it every 4-8 hours of importing.

PTAgg requires Postgres to do most of the work for it. Postgres requires CPU & disk resources. If you are importing lots of hands in the background, that can take up quite a bit of CPU & Disk.

As for slowing down the more hands it has, that is true of all databases.

Backup up the PTAgg database should be very quick? Its only 8 MB, should be MUCH faster than backing up a PT database.

Increasing the batch size made it about 50-100% faster on my machine. Not sure why you didn't see any increase.

-Ben

[Thoth]

Small question about subsriptions. I'm just moving up to 2/4 at the moment, but hope to move up again in a few months.

If I take a 6-month Silver subscription would it be possible to upgrade that to Gold when I move up to 3/6 after say 2 months?

[SixthSense]

[ QUOTE ]
Small question about subsriptions. I'm just moving up to 2/4 at the moment, but hope to move up again in a few months.

If I take a 6-month Silver subscription would it be possible to upgrade that to Gold when I move up to 3/6 after say 2 months?

[/ QUOTE ]

Sorry, but our system isn't very fancy. You have to cancel a subscription and then start a new one. For that reason, if you plan on moving up soon, just use the 1-month billing option.

-Ben

[goodguy_1]

[ QUOTE ]
PTAgg has a known memory leak, so its best to shut down the app and restart it every 4-8 hours of importing

[/ QUOTE ]
I'm importing after a reboot and running no other applications and I'm only able to import ~12K per hour-this is not after 4-8 hours this is the first hour after a fresh reboot

[ QUOTE ]
PTAgg requires Postgres to do most of the work for it. Postgres requires CPU & disk resources. If you are importing lots of hands in the background, that can take up quite a bit of CPU & Disk

[/ QUOTE ]
you seem to be avoiding this issue. In every post on this topic I've said I'm importing after a fresh reboot and running no other applications-definetly not importing hands in the background-that would be completely untenable with its current slowness.

Maybe I'm just being too ambitious with all the data I have. I can always cut back the amount of data to utilize-but this is a major issue for advanced users that have access to alot of mined data-not hundreds of thousands of hands but mulitple very large db's of 2M+ hands.

[ QUOTE ]
As for slowing down the more hands it has, that is true of all databases.


[/ QUOTE ]
yes any experienced PT user knows that but the ptagg database importing has crawled to 15K per hour after about 4M hands total aggregated.

When I first started aggregating hands i recall the rate was at ~30K-40K hands per hour.

Is it possible to have more than one ptagg db? or a similar workaround. I know obviously you only want to have one aggregated database but perhaps there are capacity issues for larger amounts of data?

I'm just guessing that the ptagg database may not be cut out for this much data -if it was then why is it slowing down so much? regular psql db's dont slow down like this.

In large psql PokerTracker daatbases the rate at which one can import hands doesnt deteriorate like this.

With ptagg when you add a large amount of data the rate you can import new hands is severly deteriorated..this doesnt happen in large regular psql db's I know I've worked with and currently have mulitple 4,5,6Million hand databases.

[ QUOTE ]
Backup up the PTAgg database should be very quick? Its only 8 MB, should be MUCH faster than backing up a PT database


[/ QUOTE ]
oh yea backing up this database takes like 5 minutes.A regular db that has processed same number of hands ie 4M would take about 45 minutes. I never said I had an issue with backing ptagg up.

I just mentioned backing it up because importing into ptagg is an extremely time consuming affair if you have alot of data and dont want to have to go thru it multiple times.
[ QUOTE ]
Increasing the batch size made it about 50-100% faster on my machine. Not sure why you didn't see any increase.

[/ QUOTE ]

this hasnt healped me at all seriously. Problem is we the users are working with ptagg db that have millions of hands and you havent yet tested it with that much data.

I saw a prior post on this where another poster mentioned how slow importing was and you seemed to do the same thing you did in reference to my post ie you are not acknowleding this is a serious issue-please stop the denial and reposting the same replies to other hardcore users.

This issue is not enough to change my mind about how great a tool this is ..I can adapt and cut back my ambitions but you need to stop denying that this is an issue.

I'm currently trying to aggregate/import hands at a rate of 10K per hour. For the remaing 2-3 db's I wanted to aggregate it will take me 340 hours at this rate.