Quick scan for signal🙏🏻 Hey TV, this is QSFS, following:
^^ Quick scan for drift (QSFD)
^^ Quick scan for cycles (QSFC)
As mentioned before, ML trading is all about spotting any kind of non-randomness, and this metric (along with 2 previously posted) gonna help ya'll do it fast. This one will show you whether your time series possibly exhibits mean-reverting / consistent / noisy behavior, that can be later confirmed or denied by more sophisticated tools. This metric is O(n) in windowed mode and O(1) if calculated incrementally on each data update, so you can scan Ks of datasets w/o worrying about melting da ice.
^^ windowed mode
Now the post will be divided into several sections, and a couple of things I guess you’ve never seen or thought about in your life:
1) About Efficiency Ratios posted there on TV;
Some of you might say this is the Efficiency Ratio you’ve seen in Perry's book. Firstly, I can assure you that neither me nor Perry, just as X amount of quants all over the world and who knows who else, would say smth like, "I invented it," lol. This is just a thing you R&D when you need it. Secondly, I invite you (and mods & admin as well) to take a lil glimpse at the following screenshot:
^^ not cool...
So basically, all the Efficiency Ratios that were copypasted to our platform suffer the same bug: dudes don’t know how indexing works in Pine Script. I mean, it’s ok, I been doing the same mistakes as well, but loxx, cmon bro, you... If you guys ever read it, the lines 20 and 22 in da code are dedicated to you xD
2) About the metric;
This supports both moving window mode when Length > 0 and all-data expanding window mode when Length < 1, calculating incrementally from the very first data point in the series: O(n) on history, O(1) on live updates.
Now, why do I SQRT transform the result? This is a natural action since the metric (being a ratio in essence) is bounded between 0 and 1, so it can be modeled with a beta distribution. When you SQRT transform it, it still stays beta (think what happens when you apply a square root to 0.01 or 0.99), but it becomes symmetric around its typical value and starts to follow a bell-shaped curve. This can be easily checked with a normality test or by applying a set of percentiles and seeing the distances between them are almost equal.
Then I noticed that on different moving window sizes, the typical value of the metric seems to slide: higher window sizes lead to lower typical values across the moving windows. Turned out this can be modeled the same way confidence intervals are made. Lines 34 and 35 explain it all, I guess. You can see smth alike on an autocorrelogram. These two match the mean & mean + 1 stdev applied to the metric. This way, we’ve just magically received data to estimate alpha and beta parameters of the beta distribution using the method of moments. Having alpha and beta, we can now estimate everything further. Btw, there’s an alternative parameterization for beta distributions based on data length.
Now what you’ll see next is... u guys actually have no idea how deep and unrealistically minimalistic the underlying math principles are here.
I’m sure I’m not the only one in the universe who figured it out, but the thing is, it’s nowhere online or offline. By calculating higher-order moments & combining them, you can find natural adaptive thresholds that can later be used for anomaly detection/control applications for any data. No hardcoded thresholds, purely data-driven. Imma come back to this in one of the next drops, but the truest ones can already see it in this code. This way we get dem thresholds.
Your main thresholds are: basis, upper, and lower deviations. You can follow the common logic I’ve described in my previous scripts on how to use them. You just register an event when the metric goes higher/lower than a certain threshold based on what you’re looking for. Then you take the time series and confirm a certain behavior you were looking for by using an appropriate stat test. Or just run a certain strategy.
To avoid numerous triggers when the metric jitters around a threshold, you can follow this logic: forget about one threshold if touched, until another threshold is touched.
In general, when the metric gets higher than certain thresholds, like upper deviation, it means the signal is stronger than noise. You confirm it with a more sophisticated tool & run momentum strategies if drift is in place, or volatility strategies if there’s no drift in place. Otherwise, you confirm & run ~ mean-reverting strategies, regardless of whether there’s drift or not. Just don’t operate against the trend—hedge otherwise.
3) Flex;
Extension and limit thresholds based on distribution moments gonna be discussed properly later, but now you can see this:
^^ magic
Look at the thresholds—adaptive and dynamic. Do you see any optimizations? No ML, no DL, closed-form solution, but how? Just a formula based on a couple of variables? Maybe it’s just how the Universe works, but how can you know if you don’t understand how fundamentally numbers 3 and 15 are related to the normal distribution? Hm, why do they always say 3 sigmas but can’t say why? Maybe you can be different and say why?
This is the primordial power of statistical modeling.
4) Thanks;
I really wanna dedicate this to Charlotte de Witte & Marion Di Napoli, and their new track "Sanctum." It really gets you connected to the Source—I had it in my soul when I was doing all this ∞
Efficiency
KERPD Noise Filter - Kaufman Efficiency Ratio and Price DensityThis indicator combines Kaufman Efficiency Ratio (KER) and Price Density theories to create a unique market noise filter that is 'right on time' compared to using KER or Price Density alone. All data is normalized and merged into a single output. Additionally, this indicator provides the ability to consider background noise and background noise buoyancy to allow dynamic observation of noise level and asset specific calibration of the indicator (if desired).
The basic theory surrounding usage is that: higher values = lower noise, while lower values = higher noise in market.
Notes: NON-DIRECTIONAL Kaufman Efficiency Ratio used. Threshold period of 30 to 40 applies to Kaufman Efficiency Ratio systems if standard length of 20 is applied; maintained despite incorporation of Price Density normalized data.
TRADING USES:
-Trend strategies, mean reversion/reversal/contrarian strategies, and identification/avoidance of ranging market conditions.
-Trend strategy where KERPD is above a certain value; generally a trend is forming/continuing as noise levels fall in the market.
-Mean reversion/reversal/contrarian strategies when KERPD exits a trending condition and falls below a certain value (additional signal confluence confirming for a strong reversal in price required); generally a reversal is forming as noise levels increase in the market.
-A filter to screen out ranging/choppy conditions where breakouts are frequently fake-outs and or price fails to move significantly; noise level is high, in addition to the background buoyancy level.
-In an adaptive trading systems to assist in determining whether to apply a trend following algorithm or a mean reversion algorithm.
THEORY / THOUGHT SPACE:
The market is a jungle. When apex predators are present it often goes quiet (institutions moving price), when absent the jungle is loud.
There is always background noise that scales with the anticipation of the silence, which has features of buoyancy that act to calibrate the beginning of the silence and return to background noise conditions.
Trend traders hunt in low noise conditions. Reversion traders hunt in the onset of low noise into static conditions. Ranges can be avoided during high noise and buoyant background noise conditions.
Distance between the noise line and background noise can help inform decision making.
CALIBRATION:
- Set the Noise Threshold % color change line so that the color cut off is where your trend/reversion should begin.
- Set the Background Noise Buoyancy Calibration Decimal % to match the beginning/end of the color change Noise Threshold % line. Match the Background Noise Baseline Decimal %' to the number set for buoyancy.
- Additionally, create your own custom settings; 33/34 and 50 length also provides interesting results.
- A color change tape option can be enabled by un-commenting the lines at the bottom of this script.
Market Usage:
Stock, Crypto, Forex, and Others
Excellent for: NDQ, J225, US30, SPX
Market Conditions:
Trend, Reversal, Ranging
Price Density (Market Noise) by Alejandro PThis is an indicator based on the Price Density concept. Price Density is a quantifiable method of measuring market noise for a certain period of candles.
This indicator also has the option to use relative percentile values which transforms the indicator from an absolute value measure to a percentile based measure so it can be more easily compared across vastly different assets.
We can use Price Density to set rules and only trade particular systems when noise is at an appropriate level. For example, if noise is high then we would want to avoid trend following strategies and instead trade mean-reversion strategies, and vice-versa when the opposite is true. Price Density can also be used to match assets to strategies. Some assets will be naturally more noisy than others and therefore we might have a principle where we only trade those noisy assets with our mean reversion strategies and the more quiet assets with trend following strategies.
Price Density can be used by looking at indicator and seeing how efficiently the indicator line moves from one side to the other. Is it in a straight line? Or are there lots of zigs and zags? The straighter the line, the less market noise there is.
Calculations:
Price Density = Sum(ATR(1), Length) / (Max(High) - Min(Low))
The numerator on top looks at the high of each of the bars, the low of a bar, calculates the difference and then sums up all of those values for the time period. This gives us an idea of the fluctuation of the individual price moves. Next, the denominator uses the maximum value from all of the highs of those bars and subtracts the minimum low out of all of the bars. This gives us the range or the height of the box and so if there's a lot of price fluctuation compared to the actual range this means we'll have a high value which gives the indication that we have high levels of noise. If however those individual fluctuations are small compared to the range this means we'll get a low value for the price density and therefore have low levels of noise.
Combo Backtest 123 Reversal & PFE (Polarized Fractal Efficiency) This is combo strategies for get a cumulative signal.
First strategy
This System was created from the Book "How I Tripled My Money In The
Futures Market" by Ulf Jensen, Page 183. This is reverse type of strategies.
The strategy buys at market, if close price is higher than the previous close
during 2 days and the meaning of 9-days Stochastic Slow Oscillator is lower than 50.
The strategy sells at market, if close price is lower than the previous close price
during 2 days and the meaning of 9-days Stochastic Fast Oscillator is higher than 50.
Second strategy
The Polarized Fractal Efficiency (PFE) indicator measures the efficiency
of price movements by drawing on concepts from fractal geometry and chaos
theory. The more linear and efficient the price movement, the shorter the
distance the prices must travel between two points and thus the more efficient
the price movement.
WARNING:
- For purpose educate only
- This script to change bars colors.
Bryant Adaptive Moving Average@ChartArt got my attention to this idea.
This type of moving average was originally developed by Michael R. Bryant (Adaptrade Software newsletter, April 2014). Mr. Bryant suggested a new approach, so called Variable Efficiency Ratio (VER), to obtain adaptive behaviour for the moving average. This approach is based on Perry Kaufman' idea with Efficiency Ratio (ER) which was used by Mr. Kaufman to create KAMA.
As result Mr. Bryant got a moving average with adaptive lookback period. This moving average has 3 parameters:
Initial lookback
Trend Parameter
Maximum lookback
The 2nd parameter, Trend Parameter can take any positive or negative value and determines whether the lookback length will increase or decrease with increasing ER.
Changing Trend Parameter we can obtain KAMA' behaviour
To learn more see www.adaptrade.com
Efficient Auto LineMore Efficiency
Based on the Auto-Line code, the Efficient Auto Line aim to provide a more controlled adaptivity of the indicator. The first indicator of this sort worked this way : when the absolute difference between the price and the indicator is higher than the previous indicator +/- A pips of amplitude, the indicator will display the closing price, else its anterior value. The second indicator (Auto-Line) was adaptive and used the standard deviation instead of a constant A . This indicator will run both methodology providing both a trend strength indicator (Efficiency Ratio) parameter and two constant parameter.
Parameters
The length parameter will control the period of the efficiency ratio, a high period return lower values of the efficiency ratio. Since its an indicator in a range of (0,1) we use it to make our indicator more adaptive in trending market, this is when we need our two constant parameters, the fast/slow parameter can be any amount of pips where fast < slow , when the price is trending (efficiency ratio close to 1) the indicator will use the fast parameter, if its ranging (efficiency ratio away from 1) the indicator will use the slow parameter, then it will work like the first methodology previously explained. So the fast parameter should be equal to a small movement of pips (0.0001 or 1 pip) and the slow parameter should be equal to a number of pips you wont expect to see in a ranging market. At this point it is good to test for both parameter and see which values work better (a more automatic process is in development) .
Hope you like it !
Efficient PriceTrading The Movements That Matters
Inspired by the Price Volume Trend indicator the Efficient Price aim to create a better version of the price containing only the information a trend trader must need.
Calculation
This indicator use the Efficiency Ratio as a smoothing constant, it is calculated as follow :
ER = abs(change(close,length))/sum(abs(change(close)),length)
The goal of the Efficiency Ratio is to show if the market is trending or ranging.If ER is high then the market is considered to be trending, if ER is low then the market is considered to be ranging.
Then the Efficient Price is calculated :
EP = cum(change(close)*ER)
When the price is trending, the indicator will show movements of the price with unchanged volatility, but if the price is not trending then the indicator will flatten those movements.Think of this indicator as both a filter and a compressor and the Efficient Price as some kind of threshold.
The Efficient Price As Input For Indicators/Strategies
If the indicator show the movement of the trending price, it can be interesting to use it as input in order to reduce the number of false signals in a strategy.
We will test 2 MACD strategy provided by tradingview, one using the closing price (In Red) and one with the efficient price (In White) as input
with both the following parameters :
fastLength = 50
slowlength = 200
MACDLength = 20
length = 50
Where length is the parameter of the Efficient Price.A spread of 2 pips is used.
Without Efficient Price : 26.88% of profitability, 69 pips of profit.
With Efficient Price : 38.46% of profitability, 336 pips of profit.
The difference of profitability is of 11.58%, the strategy with the Efficient Price made few trades and its equity have a lower variance than the equity of the MACD strategy using closing price.
Smoothed Version
It is possible to smooth the indicator output by using the following code :
EP = cum(change(close,length)*ER)
Hope you enjoy
For any questions/demands feel free to pm me, i would be happy to help you
PFE (Polarized Fractal Efficiency) The Polarized Fractal Efficiency (PFE) indicator measures the efficiency
of price movements by drawing on concepts from fractal geometry and chaos
theory. The more linear and efficient the price movement, the shorter the
distance the prices must travel between two points and thus the more efficient
the price movement.
FREE INDICATOR: POLARIZED FRACTAL EFFICIENCYLooking for something other than a moving average to help determine not only a trend's strength, but also it's direction? Try PFE!
PFE was developed by Hans Hannula that was invented to determine price efficiency over a user-defined time period.
The Polarized Fractal Efficiency indicator is, in the essence, an exponentially smoothed ratio of the length of two lines: (1) of a straight line between today’s close and the close Period days ago, and (2) of a broken line connecting all Close points between today and Period days ago. The indicator output varies between -100 and 100. The theory behind this indicator is that if it is >50 (or <-50) then the market is likely to reverse its trend from positive to negative (or from negative to positive).
Other usage:
Securities with a PFE greater than zero are deemed to be trending up, while a reading of less than zero indicates the trend is down. The strengh of the trend is measured by the position of the PFE relative to the zero line. As a general rule, the further the PFE value is away from zero, the stronger and more efficient the given trend is. A PFE value that fluctuates around the zero line could indicate that the supply and demand for the security are in balance and price may trade sideways.
As with all indicators, finding something that works well along side this would be the most beneficial way to use it.
Perhaps something like the Choppiness Index (related idea below) could do the trick.
Grab the source code here: pastebin.com
Installation video by @ChrisMoody here : blog.tradingview.com
