Market Model Thingy

[Jimbo]

[ QUOTE ]
(ie. How much the entire test market went up)


[/ QUOTE ]

OK, got you (I think), it might be more useful to compare it to say how much the S&P 500 went up over the same timeframe or the DJIA. After all noone buys stocks randomly, at least I hope not.

Jimbo

In case any of you random buyers are following this thread please buy as much Garmin today as you are able to do so, at random intervals of course. [img]/images/graemlins/smile.gif[/img]

[CallMeIshmael]

[ QUOTE ]
[ QUOTE ]
Phone,

"What kind of assumptions are you using for execution prices?"

Sorry, what is an execution price?


[/ QUOTE ]

What prices are you buying and selling at?

[/ QUOTE ]


Well, I never really had a specific trading rule in mind for this model. I was more interested if these sorts of predictions were even possible. I'm almost certainly going to put some real effort into a much better model, using intraday data. There, I'll be very interested in maximizing return, and producing a solid trading rule.


Given that it seems haakee implied $10/trade is a reasonable transaction fee (and assuming a trade = buying AND selling, and we dont pay for both), I'll probably compute some returns using a strategy of buying at opening price and selling at closing price for each day.

How reasonable is the assumption you can buy/sell at open/close prices?

[Yobz]

CMI: I would also try your model by selling the stock at the opening price on the next day. Lots of companies report earnings overnight and you might be losing lots of EV this way.

Note: I'm also fairly uneducated about stocks, so I might be completely wrong. :-)

[CallMeIshmael]

[ QUOTE ]
CMI: I would also try your model by selling the stock at the opening price on the next day. Lots of companies report earnings overnight and you might be losing lots of EV this way.

[/ QUOTE ]


ohh wow. I didnt even know they were different. I just assumed each day's closing price was the next opening price. So, all calculations made were actually assuming you would sell your current day's stocks at close, and buy your next days stock also at close.



Anyway, given a $10/trade fee, I did the following simulation:

Start 10,000 accounts at $10,000 each. Get the top 15 picks for each day, then have each account randomly select 3 of those 15 picks for the day, spending 1/3 of their current worth on each of the stocks. Then, sell at close. Repeat for the 1999 days of testing. (subtract 30 dollars from the account each day)

In the end, there was about a 3.8% chance of being down, and about a 3.5% risk of ruin


These are graphs for the 25th/50th/75th percentiles for each day. I zoomed in on the first 600 days of the test in graph 2.







Obviously, these results are encouraging / seem too good to be true. Ive looked through the code again, and cant find any big mistakes. I even ran the simulation using coefficents that were trained on the same days as the ones they were tested on to make sure I didnt accidentally do that, and I got the expected blowup in results.

Also, I'd imagine the inherent assumption that you can invest large sums in a stock and still have it behave the same way it would have without that investment is pretty flawed. Even with that considered, the results still seem pretty encouraging.

[eastbay]

Lots of traps here. No guarantees you're not onto something, but not likely either. "If it was that easy everybody would do it", and all that.

Slippage and commission is important, and can turn smooth get rich quick strategies into certain losers. Looks like you're at least starting to account for commission. Don't forget slippage.

Inadvertent lookahead is an easy mistake to make. You can set up your code to help protect against it by only feeding data from a mock data source, rather than having it all in memory and just hoping you don't make a mistake in referencing the future.

You can't train and test on overlapping data. I couldn't quite tell if you were doing that from your description, but such tests are worthless for obvious implementation reasons.

If you've sure you're not doing any of the above, well, trade it by hand for awhile and see how it goes.

eastbay

[CallMeIshmael]

eastbay,


nice post. I would say Im reasonably sure there wasnt any look forward/overlap. I was pretty paranoid about this, and even reran the results with coefficients that I thought had overlap to observe that the results obtained then were a ton better (they were). Also, it was just about impossible for overlap to occur on the second test (the second 7 year period), since the testing data needs to go through a semi-longish transformation that I never did on that data set (just trained using the more recent 7 years), and that showed moderately comparable results.


Evan and I talked about ask/bid spread, which appears to be the same as slippage, yes? I had no idea what this was until a few hours ago, and appears to be something that would be v imporant to incorporate. Assume that a stock has a listed closing price of X is there a reasonable function to get an estimate of the price I would be expected to sell at, assuming I sell at close? Im learning that it is slightly smaller than X, but Im wondering if its possible to estimate it quantitatively.


"If you've sure you're not doing any of the above, well, trade it by hand for awhile and see how it goes."

Just to note: Im more interested IF its possible to using past information to predict the future, not necessarily (at least here) with trading.



Also, just as a general comment to all: I dont really know how to explain it, but you often see people on the forums with little poker knowledge arging against the common beliefs, and you cant help but think that he just doesnt understand the game. Im well aware thats what Im doing here, Id like to stress that Im skeptical of the results myself. Given the debate over the EMH, its v odd that the model could predict the top 100 performers for the next day so well; it would stand to reason that if the model were accurate, that the EMH ought to be discared, yet it hasnt been. But, again, I just couldnt find the error. If anyone knows matlab well, Id be haappy to comment/clean up the code and have them audit it.

[edtost]

[ QUOTE ]

Evan and I talked about ask/bid spread, which appears to be the same as slippage, yes? I had no idea what this was until a few hours ago, and appears to be something that would be v imporant to incorporate. Assume that a stock has a listed closing price of X is there a reasonable function to get an estimate of the price I would be expected to sell at, assuming I sell at close? Im learning that it is slightly smaller than X, but Im wondering if its possible to estimate it quantitatively.

[/ QUOTE ]

slippage and bid/ask are totally different. bid/ask is the spread between where you can buy and sell a block of shares at any given point in time. slippage is how much the market moves against you while you are executing. for very small traders, fixed commissions tend to be their biggest concern. once your account gets bigger, bid/ask becomes the dominant 'cost' to your trading. for institutional-sized accounts, slippage winds up having the largest effect.

basically, until you wouldn't be comfortable sending your trade to the exchange as a single order because it probably won't get filled, slippage isn't really something you need to worry much about. at least not until something bad happens and market liquidity dries up.

i'm sure someone who knows more than i do about single-stock trading can quote you an average bid/ask for large caps.

[ QUOTE ]
Given the debate over the EMH, its v odd that the model could predict the top 100 performers for the next day so well; it would stand to reason that if the model were accurate, that the EMH ought to be discared, yet it hasnt been. But, again, I just couldnt find the error. If anyone knows matlab well, Id be haappy to comment/clean up the code and have them audit it.

[/ QUOTE ]

EMH only needs to be discarded is your system is actually implementable in a way that would make money. trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

edit: also, to nitpick, when you do your out of sample testing, you should train your coefficients on the older data set and look at the results on the newer set.

[eastbay]

[ QUOTE ]

Evan and I talked about ask/bid spread, which appears to be the same as slippage, yes?


[/ QUOTE ]

No, slippage is the difference between your data feed's quote and your fill. In general they will not be the same.

[ QUOTE ]

Just to note: Im more interested IF its possible to using past information to predict the future, not necessarily (at least here) with trading.


[/ QUOTE ]

Then put your pen down, because the answer is yes. It has been demonstrated many times. Google Jim Simons, for example.

The EMH is approximate, just like every nice neat academic theory any complex system.

eastbay

[CallMeIshmael]

[ QUOTE ]
trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

[/ QUOTE ]

OK. Any suggestions as to how much of a linear cost?


Also, what exactly do you mean by a 1-day lag? Is using the next days opening price a more realistic sell price?

[edtost]

[ QUOTE ]
[ QUOTE ]
trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

[/ QUOTE ]

OK. Any suggestions as to how much of a linear cost?


Also, what exactly do you mean by a 1-day lag? Is using the next days opening price a more realistic sell price?

[/ QUOTE ]

25 bps? 50 bps? can someone who knows anything step in with an average bid/ask spread for equities, instead of me almost randomly guessing?

there needs to be some sort of lag between calculating your positions and trading on them. calculating based on the close on day t and trading at the open of day t+1 would be a .5 day lag, which is better than nothing. a one day lag would mean trading at the closing price on day t+1, and would probably be more realistic for someone trading daily. best would probably be to average the open and close of day t+1, implicitly assuming that you used the entire day's liquidity to make your trade.