Market Model Thingy

[CallMeIshmael]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

[/ QUOTE ]

OK. Any suggestions as to how much of a linear cost?


Also, what exactly do you mean by a 1-day lag? Is using the next days opening price a more realistic sell price?

[/ QUOTE ]

25 bps? 50 bps? can someone who knows anything step in with an average bid/ask spread for equities, instead of me almost randomly guessing?

there needs to be some sort of lag between calculating your positions and trading on them. calculating based on the close on day t and trading at the open of day t+1 would be a .5 day lag, which is better than nothing. a one day lag would mean trading at the closing price on day t+1, and would probably be more realistic for someone trading daily. best would probably be to average the open and close of day t+1, implicitly assuming that you used the entire day's liquidity to make your trade.

[/ QUOTE ]


I added to the model a discount on all sales by a factor of 0.995, and used the next days closing price as the sell price (I didnt have immediate access to the open price, and it would require a bit of leg work, so I just went next days close for now).

The results were still good, though obviously lower given the 0.995 discount.



Oddly, it appears that using the next days close price actually benefits the model.

[Phone Booth]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

[/ QUOTE ]

OK. Any suggestions as to how much of a linear cost?


Also, what exactly do you mean by a 1-day lag? Is using the next days opening price a more realistic sell price?

[/ QUOTE ]

25 bps? 50 bps? can someone who knows anything step in with an average bid/ask spread for equities, instead of me almost randomly guessing?

there needs to be some sort of lag between calculating your positions and trading on them. calculating based on the close on day t and trading at the open of day t+1 would be a .5 day lag, which is better than nothing. a one day lag would mean trading at the closing price on day t+1, and would probably be more realistic for someone trading daily. best would probably be to average the open and close of day t+1, implicitly assuming that you used the entire day's liquidity to make your trade.

[/ QUOTE ]


I added to the model a discount on all sales by a factor of 0.995, and used the next days closing price as the sell price (I didnt have immediate access to the open price, and it would require a bit of leg work, so I just went next days close for now).

The results were still good, though obviously lower given the 0.995 discount.



Oddly, it appears that using the next days close price actually benefits the model.

[/ QUOTE ]

For just sell? I think what he's saying is that if you're trading based on knowledge gathered during day 1 and day n, you need to compute returns between day n+1 and day n+2 instead of day n and day n+1 or even day n and day n+2. You should definitely not do the latter (n and n+2) if you're not normalizing the return.

[edtost]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
trading at the same price (the close) you use to make a decision is a good way to make your system unrealistic. start by modifying your program to include a one-day lag for trading and a linear transaction cost for bid/ask; if your system is still profitable after those changes, then this becomes a much more interesting discussion.

[/ QUOTE ]

OK. Any suggestions as to how much of a linear cost?


Also, what exactly do you mean by a 1-day lag? Is using the next days opening price a more realistic sell price?

[/ QUOTE ]

25 bps? 50 bps? can someone who knows anything step in with an average bid/ask spread for equities, instead of me almost randomly guessing?

there needs to be some sort of lag between calculating your positions and trading on them. calculating based on the close on day t and trading at the open of day t+1 would be a .5 day lag, which is better than nothing. a one day lag would mean trading at the closing price on day t+1, and would probably be more realistic for someone trading daily. best would probably be to average the open and close of day t+1, implicitly assuming that you used the entire day's liquidity to make your trade.

[/ QUOTE ]


I added to the model a discount on all sales by a factor of 0.995, and used the next days closing price as the sell price (I didnt have immediate access to the open price, and it would require a bit of leg work, so I just went next days close for now).

The results were still good, though obviously lower given the 0.995 discount.



Oddly, it appears that using the next days close price actually benefits the model.

[/ QUOTE ]

For just sell? I think what he's saying is that if you're trading based on knowledge gathered during day 1 and day n, you need to compute returns between day n+1 and day n+2 instead of day n and day n+1 or even day n and day n+2. You should definitely not do the latter (n and n+2) if you're not normalizing the return.

[/ QUOTE ]

I'm not sure which of the above CMI is referring to, but I agree with this.

[CallMeIshmael]

[ QUOTE ]
[ QUOTE ]
For just sell? I think what he's saying is that if you're trading based on knowledge gathered during day 1 and day n, you need to compute returns between day n+1 and day n+2 instead of day n and day n+1 or even day n and day n+2. You should definitely not do the latter (n and n+2) if you're not normalizing the return.

[/ QUOTE ]

I'm not sure which of the above CMI is referring to, but I agree with this.

[/ QUOTE ]


Yeah, I did return of (n+2)/n, which did seem a bit odd. I misunderstood what you were saying (obv at this point its pretty clear that just about any assumption regarding somebackground knowledge would be too much!). I redid it with (n+2)/(n+1), again, it went down but still with results that appear to be above the upward trend of the test market over the time period. (though, all of the additions Ive made make me a lot less certain that there isnt some error happening along the way)


Assuming that the test stat is now (n+2)/(n+1) instead of (n+1)/n, I *think* the data set could be retrained, with the obejective stat changed from (n+1)/n to (n+2)/(n+1). (ie. change the test from "given x,y,z on day n, what is an estimate for how the stock will change by close tomorrow" to "given this info, how will the stock change from close on n+1 to n+2") Is this correct, or is it making an assumption I ought not to make?


Also, semi off topic, but if this sort of thing were to be done using intraday minute by minute data, how would these additions work there?

For example, if you've decided to buy or sell at time n, is there a reasonable way to estimate a good real life price that you would actually get? Im assuming Im asking for just about an impossible task, but any input would be gladly appreciated.

Perhaps something like "average of mean low and mean close for minutes n+1 to n+10" for selling, and "average of mean high and mean close for minutes n+1 to n+10"

[edtost]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
For just sell? I think what he's saying is that if you're trading based on knowledge gathered during day 1 and day n, you need to compute returns between day n+1 and day n+2 instead of day n and day n+1 or even day n and day n+2. You should definitely not do the latter (n and n+2) if you're not normalizing the return.

[/ QUOTE ]

I'm not sure which of the above CMI is referring to, but I agree with this.

[/ QUOTE ]


Yeah, I did return of (n+2)/n, which did seem a bit odd. I misunderstood what you were saying (obv at this point its pretty clear that just about any assumption regarding somebackground knowledge would be too much!). I redid it with (n+2)/(n+1), again, it went down but still with results that appear to be above the upward trend of the test market over the time period. (though, all of the additions Ive made make me a lot less certain that there isnt some error happening along the way)


Assuming that the test stat is now (n+2)/(n+1) instead of (n+1)/n, I *think* the data set could be retrained, with the obejective stat changed from (n+1)/n to (n+2)/(n+1). (ie. change the test from "given x,y,z on day n, what is an estimate for how the stock will change by close tomorrow" to "given this info, how will the stock change from close on n+1 to n+2") Is this correct, or is it making an assumption I ought not to make?

[/ QUOTE ]

that would definitely be a reasonable thing to do. one thing to look for (in terms of how believable the model is as something other than data mining) is how much the coefficients on the various explanatory variables change when you change the specification of the model in that way.

[ QUOTE ]
Also, semi off topic, but if this sort of thing were to be done using intraday minute by minute data, how would these additions work there?

For example, if you've decided to buy or sell at time n, is there a reasonable way to estimate a good real life price that you would actually get? Im assuming Im asking for just about an impossible task, but any input would be gladly appreciated.

Perhaps something like "average of mean low and mean close for minutes n+1 to n+10" for selling, and "average of mean high and mean close for minutes n+1 to n+10"

[/ QUOTE ]

in general, i think create position at t, trade at t+1, exit at t+2 is a fairly reasonable model for any timestep delta t, in that you wouldn't want to be creating positions more often than you could trade, though i guess this could break down when you get to very small intervals.

[crazy canuck]

One easy way to check your code is to generate random data with 0 mean returns:

e.g. rand(m,n)-0.5 in matlab with m,n desired size

and see if your code generates profit (of course it shouldn't). This is not foolproof, but it's quick.

Then, you could remove stocks that have low market caps or low daily volume. Transaction cost/slippage could be fairly high for small stocks so much of your excess return could come from these.

Also, small stocks can have sick drawdowns. So if you hold a portfolio of these, you'd have to assume that some of your money is in cash. This would reduce the returns.

Also, you migh want to use pinv() instead of regress() function...sometimes it makes a difference. Same thing, just more stable numerically.

[DcifrThs]

[ QUOTE ]
One easy way to check your code is to generate random data with 0 mean returns:

e.g. rand(m,n)-0.5 in matlab with m,n desired size

and see if your code generates profit (of course it shouldn't). This is not foolproof, but it's quick.

Then, you could remove stocks that have low market caps or low daily volume. Transaction cost/slippage could be fairly high for small stocks so much of your excess return could come from these.

Also, small stocks can have sick drawdowns. So if you hold a portfolio of these, you'd have to assume that some of your money is in cash. This would reduce the returns.

Also, you migh want to use pinv() instead of regress() function...sometimes it makes a difference. Same thing, just more stable numerically.

[/ QUOTE ]

interesting...i learn new things everyday [img]/images/graemlins/smile.gif[/img]

so why exactly is pinv(X'*X) more stable numerically than (X'*X)^-1? i read the "help pinv" on it and it looks like it just calculates the inverse:

[ QUOTE ]
X = PINV(A) produces a matrix X of the same dimensions
as A' so that A*X*A = A, X*A*X = X and A*X and X*A
are Hermitian.

[/ QUOTE ]

i tested it on a few things i've done and in no case got anything different down to 4 decimal places.

also, how do i change the view so that it shows n number of decimal places?

thanks,
Barron

[crazy canuck]

The function pinv calculates the pseudoinverse:

matlab link

Sometimes the matrix inv(A'*A) has a determinant that is close to 0, so it close to non invertible. In this case there could actually be a family of vectors that minimizes the least-squares distance.

Matlab points this out by warnings...something about the condition number too high. If matlab is fine with regress (no warnings), then regress is ok.

In financial application this happens sometimes when you have inputs that are highly correlated...I guess it's ok if you have a lot of inputs, like in OP-s case.

[crazy canuck]

No idea about decimal places..if someone knows it please post it.

Up to now I just multiplied by 10 to the appropriate power.

[DcifrThs]

[ QUOTE ]
No idea about decimal places..if someone knows it please post it.

Up to now I just multiplied by 10 to the appropriate power.

[/ QUOTE ]
yea i might just search matlab index help for a while for a more permanent fix to the problem.

Barron