X – Y Plotter

One very wise (and famous) hedge fund manager told us that the first charts anyone should ever look at are scatter diagrams. Although we ourselves generally look at something else first, we agree with the importance he places on scatter diagrams and have provided that functionality.

Let's say that you are building a strategy that involves using a bond investment as an alternative to stocks. Trading individual bonds can be problematic to both research and trade, so you make the “bond” investment an Exchange Traded Fund (“ETF”). Unfortunately both IEF (the 7-10-year Treasury ETF) and TLT (the 20+-year Treasury ETF) commenced trading in 2002, which does not give you much history.

There is, however, an extremely good index created by Dr. Joseph Benning of the Chicago Board of Trade, called the Dow Jones CBT Index (Bloomberg symbol: DJCBTI) that models the 7-10 year Treasury behavior. And DJCBTI is available back to 1988. So, if we can trust the relationship between DJCBTI and IEF, we can test our strategy back to 1988. Let's see if we can trust it.

This request will take the 5-day rate of change of both DJCBTI and the close of IEF for only the dates which they have in common. If you plot them as lines, you will see their relationship over time.

Now click on , and . You will get the following template:

Select the number of panes you want and then . You will then be presented with a template with additional choices:

FDC selects the data in Column 1 of the dataset as the default for the Horizontal axis, but you may change that as you wish.

You should then enter your comparison data (by Column number) as you wish. Here we have entered Column 2 in the Dotted Line edit window:

Then click , and you will be presented with:

On the MENU line you will see . Click on that and you get:

You should then SELECT the data. You can do that in a variety of ways. First, you can just click on the data in the graph with your mouse. Alternatively, you can go to , or to the magnifying glass icon and . What you then see is the scatter diagram with the data selected:

Next, further select that portion of the whole data that you wish to analyze. You do this by clicking on the Selection Tool . With your mouse click on the picture, drag the mouse across the data you wish to analyze and click again. Following is an example of the selected data:

Once the data is further selected, you will find the button in the upper right corner of the window. Click it and select the type of fitting that you want to do. Here we have selected a linear fit:

Then . You will get the scatter plot with your line displayed. If that's what you want, then click on the Selection Tool again, and .

By Retaining New Data, you will add a third column to your dataset, such that temp1 now has 3 columns. That third column can be further analyzed as you wish. Should you click on the Note tab you will get the parameters of the linear fit, specifically the Y-intercept and the slope:

Should you have chosen a parabolic fit, the formula for which would be: a + bx + cx², you would get the values for a, b and c in that such in the Note.

What this exercise has done is to enable you to see the relationship of the two values over price. This can be significantly different than their relationship over time.

Density

Syntax: N density [dataset] , where N represents the number of histogram bars. For example:

This request takes the frequency distribution of the 5-day changes of SPX (over the last 1000 days), and puts them in 100 bins. The result is a 2-column dataset:

** N.B. The text of all commands is located at the end of this document. That way you can simply copy them over to your own command set, should you wish to duplicate or modify the research

If you then click on the plot tab you will see a histogram of the distribution:

If you click on the “Show Percentiles” icon (at the right on the icon bar), a pop-up box will appear which will give you the ability to select percentiles and choose a contrasting color.

Here's the output:

The 5-day price changes seem to be distributed about the mean in a reasonably “normal” manner. Here's a closer look at the percentile box:

Among other things, this shows you that the mean 5-day gain (the 50 Percentile) is 1.79 points. These values are also visible if you click on the note tab .

However, let's look at some conditional probabilities. For example, what is the distribution of 5-day changes of the close if the 25-day moving slope (a smooth measure of market momentum) is moving up? Here's how to make that request and what the distribution looks like:

Again, we have a distribution that looks reasonably normal. But the 50 percentile is now 12.59 points.

Furthermore, if we highlight just the positive returns you can see how your probability of a subsequent 5-day gain is greatly enhanced by using that momentum indicator. Over 70 percent of the observations are profitable.

-------------------------------

100 density 5 change cl spx last 1000

(5 change cl spx) when ((change (25 movslope cl spx back 1)) gt 0)

100 density temp1

------------------------------------------------------------------------------

Filter

With our FILTER function, you can duplicate any linear smoother imaginable. The possibilities are infinite. (Or to be ridiculously precise: any linear filter with a rational transfer function.)

Syntax: 'm1 m2 m3' filter [dataset]

in which m1, m2 and m3 are smoothing coefficients. Note that the smoothing coefficients are listed chronologically, from oldest to current. That is, m3 is the smoothing coefficient applied to the latest data (e.g. today), with m2 being applied to the previous day, and m1 being applied to two days prior to today.

This function can be used to compute moving averages and weighted moving averages. For instance, ‘.25 .25 .25 .25' filter close qqqq will produce a dataset in which each of the last 4 days are weighted equally and then added together. That request is identical to that produced by either 4 movave cl qqqq or 1 1 1 1 wtave cl qqqq. Likewise the request ‘-1 -1 -1 -1 -1 0 0 1 2 2' filter cl djia will produce an oscillator that mimics the MACD indicator used by many traders.

** N.B. The text of all commands is located at the end of this document. That way you can simply copy them over to your own command set, should you wish to duplicate or modify the research.

Alternate syntax: 'm1 m2 m3; a1 a2 a3; i1 i2 i3' filter [dataset]

in which a1, a2, and a3 are autoregressive coefficients, and i1, i2 and i3 are the autoregressive initial conditions. That is:

‘ [moving average coefficients] ; [autoregressive coefficients] ; [initial values] ' filter [dataset]

Note that the quotes and semicolons are required.

The process of exponential smoothing requires that one take say 20% of the current price, and add that to 80% of the cumulative value calculated for the previous day. For example: ‘.2; .8' filter close spx last 200 (equivalent to requesting .2 expave cl spx last 200 or 9 expave cl spx last 200) will result in a smoothing taking 20 percent of the current price of the close of the SPX, and adding it to 80 percent of the value calculated for the previous day.

That's fine, but what do you use for the previous day's value when there is no previous day, such as when you are first starting out? FDC solves that problem for you by using the first possible value of the dataset as the initial value. But that potentially creates an adjustment problem if you have (by virtue of your smoothing) created values not close in scale to the original dataset. That's why we give you the opportunity to specify an initial value. If you do not specify an initial value, FDC will use the default value.

In the example immediately above there is no change in scale, so FDC's supplying of the initial value is perfect. However, should you enter ‘.2; .6' filter close spx last 200, the scale will be different from that of the price series, and you should supply an initial value. '.2; .6; 600' filter close spx last 200 would be a good choice, but that's up to you.

A second example: '.3 .2; .4 .1' filter close spx last 200 does the following:

It takes .2 of the current price added to .3 of yesterday's price and then added to .1 of yesterday's indicator and .4 of the prior day's indicator. That is, this is a 2-stage version of an exponential moving average. The chart below shows that value when overlaid onto the SPX bar chart. You may have as many versions of this as you wish.

A third example: Should you specify '; 1 1 ; 1 1' filter date spx last 100, you will get an interesting response. First, there are no moving average coefficients, so nothing actually gets done to the dataset [date spx last 100], such that the choice of the dataset is irrelevant. But here's what does happen: For the initial values of 1 and 1, it continues to add the latest number to the previously determined value. Here's what the results look like:

The screenshot on the left is the beginning of the series, which many will recognize, and the one on the right is the end of the series (100 days later). If you were then to request

, a 1-day rate of change of the series, you would get the following:

That is, when you create this series, and take the rate of change of each value to the previous value, the ratio converges to 0.618…., otherwise known as “phi” or the Golden Ratio. We do not suggest that has any value, but use the process to illustrate functionality.

You can see by the previous example that if the coefficients of the moving averages and auto-regressive factors add up to greater than 1, (in that case they add up to 2), you are going to get eventual values significantly larger than that of the dataset. Likewise, if those coefficients add up to less than 1, the eventual values will be smaller than the dataset. When they add up to 1, you will get values that approximate the dataset. So if you use the filter function to smooth data, have your coefficients add up to 1.

Fourth example: By mixing coefficients that produce smoothers together with coefficients that produce oscillators, you will get a chimera with characteristics of both. If this were a biology experiment it might be boycotted by various groups, but here we are okay. Try: '.1 .1 .1 .1 .1 .2 .3; -.1 -.1 -.1 -.1 -.1 0 0 .1 .2 .2' filter cl spx last 100

This produces the following when graphed with the SPX dataset:

-------------------------------

'm1 m2 m3' filter [ds]

‘.25 .25 .25 .25' filter close qqqq

4 movave cl qqqq

1 1 1 1 wtave cl qqqq

‘-1 -1 -1 -1 -1 0 0 1 2 2' filter cl djia

‘.2; .8' filter close spx last 200

.2 expave cl spx last 200

9 expave cl spx last 200)

‘.2; .6' filter close spx last 200

'.2; .6; 600' filter close spx last 200

'.3 .2; .4 .1' filter close spx last 200

reset

'; 1 1 ; 1 1' filter date spx last 100

1 roc temp1

'.1 .1 .1 .1 .1 .2 .3; -.1 -.1 -.1 -.1 -.1 0 0 .1 .2 .2' filter cl spx last 100

------------------------------------------------------------------------------

Horizontal Sorting - the HSORT function

You would like to pursue a sector rotation strategy, and need to rank all of the assets according to a particular criterion. First let's create a dataset with all of the assets being ranked. There may be hundreds of such assets, and we do not want to enter them all manually, so let's set up a loop that pulls all of the datasets out of a list, and stacks them side-by-side according to dates they have in common.

Here's the list of Dow Jones European stock indices, identified as “Euro_industry_sectors”

Then here's the loop construction:

** N.B. The text of all commands is located at the end of this document. That way you can simply copy them over to your own command set, should you wish to duplicate or modify the research. A line-by-line explanation of the operation of the loop is also at the end of this document. Graphics Memory Warning: The operations here perform lots of calculations and generate enormous output. Should you do this yourself, do not have all of the output open at the same time. Doing so is likely to exhaust your Windows Graphics Memory, which is typically much smaller than regular memory.

Here's what the result looks like.

Should you enter “dj_euro_stoxx_sectors first 1”, you will learn that the first entry that is common to all of the sectors is 19911231, or December 31, 1991. Should you enter “length dj_euro_stoxx_sectors” you will see that the dataset has 3441 rows (daily dates). This is obviously enough data for a valid test.

Next we need to produce the measurement by which we are going to rank the sectors. Separately one of our principals has written articles recommending the moving slope rate of change (MSROC) as an excellent momentum indicator for such an exercise. MSROC has the advantages of being timely, smooth and price-independent. Let us then calculate a 63-day MSROC and a 21-day MSROC for each sector, and average those two MSROCs. Note that there are typically 21 trading days in a month and 63 trading days in a quarter. Thus we are looking to rank the sectors according to the average of monthly and quarterly momentum.

Here's the output:

** N.B. To keep the column labels consistent, we have supplied text at the end for applying column labels.

Before we go about picking our strongest sector, we should consider the possibility that all equities will be below our investment threshold. That is, is it possible that we would not want to be in equities at all, perhaps because they were all declining in value? Here we are going to recommend that if the yield on cash is greater than the yield on equities, you move to cash. But let's go one step further and place cash in higher esteem by comparing its 21-day MSROC to the combined longer-term MSROC of the equity sectors. The yield on cash will be the 3-month Euribor rate (substituting the 3-month German rate before the existence of the Euro).

Above is shown the dataset “eurocashrate”, which is expressed as a yield. What we need is that dataset expressed as an accumulating asset – as though it were a money market fund whose value compounded daily. It can then be compared to our sectors, which are also expressed as accumulating assets. Here's what to enter:

The top line merely calculates the daily value at which the eurocashrate compounds (on the basis of 252 trading days per year). The second line then compounds them over the full length of the data. It is the algebra of compounding interest rates. The third line calculates the 21-day MSROC of that accruing value, and the last line glues our cash MSROC onto our dataset with the MSROCs of the stock sectors, creating a new dataset and giving it a much shorter name.

Now that we have MSROCs for each asset (the stock sectors and cash), we need to see how they rank. If you request “hsort desmc”, you will get a dataset with the data rearranged in the 19 columns from lowest to highest. If you have forgotten how many columns you have, just request “width desmc”.

Seeing the rankings are unnecessary; what we really need is to identify the MSROC value of the highest ranked asset. The statement solves our needs.

Again, the text (hsort desmc) sorts the values in each row and rearranges them in columns with the lowest value in column 1 and the highest value in the last column. If there are 19 columns (as there are in this example), the winning value will be in the 19^th column of this dataset. After we have sorted the data such that the highest value is in column 19, we have then requested only column 19. The output will be a 1-column dataset with that highest MSROC value:

Note, if you did not know the number of columns, you could request:

Next, you just need to know which of our 19 columns of the dataset desmc has that value:

This statement identifies the condition when for each day the value of any column of the dataset desmc equals the value of highest_column_value. When that condition exists, the resulting dataset will have a value of 1 in that column, with values of 0 in all of the other columns. In the part of the dataset illustrated below, column 6 is ranked first, but then the winner switches to column 2.

Okay, we now know the sectors in which we should be invested, and when (more on this soon). We next need to calculate what we will earn from those investments. To calculate the daily return of each stock sector asset:

That's only the equity sectors; don't forget the cash returns:

Now we want to make this as realistic a test as possible, so we have to account for a lag of 1-day between when we learn which sector is highest ranked, and when we can acquire it.

By multiplying, we produce returns of zero in 18 of our 19 columns, with only a return showing in the winning column. But we still have 19 columns, and need only one.

hsum is the horizontal sum function, which adds all of the columns horizontally, producing one column with their sum.

This gives you daily returns. What you next need to do is to compound those:

The function cumprod takes the cumulative product of each row multiplied by all those going before it.

This worked great in a generally rising market. Unfortunately the only asset it could employ when the overall market was in decline was cash. That in and of itself illustrates the advantages of long/short programs over long-only programs.

What we have shown here is a fairly simple sector rotation study. A few basic modifications will vastly improve the results. For example, the moving slope rate of change was chosen because it has the good characteristics of being both smooth and timely. However, smoothing the raw data prior to performing our moving slope calculations, and additional smoothing after (but still prior to sorting) generated results an order of magnitude greater.

-------------------------------

reset

:for #1 :in Euro_industry_sectors

:if (count 1) = 1

dummy gets #1 col 1

:else

dummy gets (dummy common #1 col 1)

:endif

:endfor

dj_euro_stoxx_sectors showgets dummy

-------------------------------

msroc63: 63 msroc dj_euro_stoxx_sectors

msroc21: 21 msroc dj_euro_stoxx_sectors

dj_euro_stoxx_msroc_comb showgets (msroc63 + msroc21)/2

-------------------------------

Column labeling:

'sx3p,sx4p,sx6p,sx7p,sx8p,sxap,sxdp,sxep,sxfp,sxip,sxkp,sxmp,sxnp,sxop,sxpp,sxqp,sxrp,sxtp' setlabels dj_euro_stoxx_msroc_comb

-------------------------------

dailycashreturn: (1+(eurocashrate/100))^(1/252)

accrue: 100* cumprod dailycashreturn

accrual21msroc: 21 msroc accrue

desmc: dj_euro_stoxx_msroc_comb common accrual21msroc

-------------------------------

highest_column_value: (hsort desmc) col 19

highest_column_value: (hsort desmc) col (width desmc)

sector_winner: desmc = highest_column_value

-------------------------------

equity_gains: dj_euro_stoxx_sectors / (dj_euro_stoxx_sectors back 1)

allgains: equity_gains common dailycashreturn

-------------------------------

combined_gains: (allgains ahead 1) * sector_winner

daily_switch_returns: hsum combined_gains

compound_switch_returns: cumprod daily_switch_returns

-------------------------------

Explanation of the loop:

Line 1: reset any temporary datasets

Line 2: Starts the first loop (#1), calling up the list named “Euro_industry_sectors”. The loop then performs computation in the lines between :for and :endfor sequentially on all dataset in the list Euro_industry_sectors.

Line 3: Line 3 involves the loop function count. It tells you the number of the current value of #1 in the list. That is, #1 represents each name in a list successively. There might be 20 or 200 names in that list. If #1 currently holds the name of the third dataset in the list, then “count 1” would be 3. If #1 currently holds the name of the ninety-eighth dataset in the list, then “count 1” would be 98. If you were also using the variable #2, you would access its count by “count 2”. Thus line 3 is only true when #1 contains the name of the first item in the list.

Line 4: then create a new dataset called “dummy”, and dummy will consist of Column 1 of the first item in your list.

Line 5: If your count in that list is not 1,

Line 6: then take column 1 of the next item in your list and put it together with dummy and rename the new dataset “dummy”.

Line 7: Ends the “if” condition.

Line 8: Ends the loop when all items in the list are considered.

Line 9: Creates (and saves) a new dataset called “dj_euro_stoxx_sectors” from dummy, and displays it.

------------------------------------------------------------------------------

Wavelet De-noising – the “multiscale” function

Wavelets are short ephemeral cyclic behavior. If you would like to know more about them, we recommend the books listed at the end of this section. If you would still like to know more, contact us, and we will give you a bigger bibliography. Those of you who are familiar with signal processing literature will find our multiscale function to be essentially a causal version of maximum overlap discrete wavelet transform. A “true” wavelet transform would use symmetric data – past and future data. Unfortunately traders do not have the luxury of knowing future data. Thus in this version the transform has been shifted to use only past and present data.

FDC permits use of five different wavelet transforms:

Haar (‘H')

Daubechies 4 (‘D4')

Daubechies 5 (‘D5')

Coiflets 6 (‘C6')

Scalar 5 (‘S5')

Each of these transforms operates on one column of data and produces 6 columns of output. The syntax is:

‘T' multiscale [dataset], in which T represents the transform.

Note that the quotes are necessary, and the function is case sensitive. Let's create an example:

generates the following output:

** N.B. The text of all commands is located at the end of this document. That way you can simply copy them over to your own command set, should you wish to duplicate or modify the research

This output when graphed would look as follows:

Note that we have plotted them in separate panes because of their differing scales, and to avoid confusion. The first column is the shortest term.

We specifically recommend these transforms as neural network inputs. However, an interesting demonstration can be made on the above dataset. Add together the four shortest-term outputs, and compare them to the raw data:

This data has a scale significantly smaller than that of the put/call ratio, so let's turn the latter into an oscillator by taking its current value minus a long-term average:

Put/call data is usually smoothed with a 21-day moving average to reduce the known monthly cyclicality caused by options expiry. (There are typically 21 trading days in a month.) Thus the next step is to average both of our datasets:

Here's the output, with our 4-wavelet composite in Blue and the oscillator version of the put/call ratio in Red. As you can see, Blue leads Red significantly, without distorting any patterns.

This is exactly the opposite of what most traders believe. That is, most traders believe that shorter-term data is noise, which must be removed and discarded to get at the “true signal”. What we have illustrated here is that the shorter-term data actually becomes a significant part of the signal, and that elimination of the longer-term data gives you quicker answers as to what's going on. This should not surprise those who understand market data to be fractal, with all price time frames exhibiting both signal and noise. So do not discard that shorter-term data.

-------------------------------

pcwave: 'H' multiscale putcallratio

pcwave4: (hsum pcwave cols 1 2 3 4)

pcratio_oscillator: pcratio - (50 movave pcratio)

21 movave (pcwav4, pcratio_oscillator)

-------------------------------

Introductory References:

C. Sidney Burrus, Ramesh A. Gopinath, and Haitao Guo

Introduction to Wavelets and Wavelet Transforms, A Primer

Barbara Burke Hubbard

The World According to Wavelets, Second Edition

------------------------------------------------------------------------------