Seminar – Research in endurance (Tevis study cont)
| March 25, 2014 | Posted by Melinda under Uncategorized |
It’s been a whirl wind week since we chatted. I’m getting a taste of clinics and it’s boring, busy, hectic, frightening, etc. But, we aren’t here to talk about dairy cows and what the industry is or isn’t doing….so let’s get back to the conference.
Brief recap from the previous post: Remember that…
1. …chloride is king when it comes to possibly predicting future failure/success later on in the ride.
2. …with the exception of possibly heart rate, physical exam (PE) parameters during the vet check are not good predictors for success/failure later on in the ride.
The real question is: is chloride a better predictor than HR? If not….then there is no point in continuing to pull blood and explore chloride if the very non-invasive heart rate is as good or better.
That is what the Tevis studies in 2012 and 2013 set out to look at.
2012 study
2012 was considered a “pilot study”. In general it seemed to confirm everything we already discussed in the previous post – Chloride is “king” and is a better predictor of failure as compared to other parameters. Total protein was a close follow up.
- The conclusion was that horses with increased dehydration, and increased elyte loss had an increase in failure rate.
- None of the PE factors were important.
2013 study
Even though the 2012 and 2013 studies were essentially the same design, the 2013 study has an important difference – participation wasn’t REQUIRED – it was strictly voluntary whether you wanted blood pulled at Robinson.
This introduces a selection bias into the data of 2013 that was not present in the 2012 study.
Something very interesting occurred (at least I think it’s interesting….) in 2013 based on people being able to opt out.
- In 2013, those people that ended up in the rider option group were very unlikely to have blood drawn at Robinson.
- Those that were pulled over time were also over represented in the “did not have blood drawn” group.
So what’s a good predictor of failure?
The physical exam
Not reported factors (ie – stuff that wasn’t recorded on the vet card) was significantly associated with failure. Remember that the stats are only being evaluated at Robinson Flat (about 1/3 of the way through the 100 mile ride) and then being used to predict failure at later points in the ride. So this isn’t a case of a vet seeing a lame horse and then not bothering to record the rest of the parameters – these are horses that are being cleared to continue on in the ride that are missing “stats” on their cards at this point – such as jug fill, anal tone. There was some discussion on why this might be – perhaps the vets are concerned with something else on the PE that is occupying more of their attention.
The worst predictors (ie, not helpful at ALL) are skin tent, muscle tone, tack galls. The list doesn’t surprise me. If you could have asked me what I thought the worst predictor was in the PE exam of future failure, I would have named skin tenting. SO variable. Muscle tone makes sense to me as well, considering how subjective it is depending on the vet scoring it.
The next worst predictors were back, cap refill, gait, gut sounds, attitude, mucus membranes, overall impression. In a bigger study some of these might result in being significant, especially the ones at the end of the list (they are listed in order of worst to not so worse…). I asked whether a power study had been done (let’s you know how many subjects you need to get significant results at the end) and while he didn’t know, when I asked for a “gut feeling” he said 400-500 horses. (Tevis has about 200 horses).
The best predictors on the PE for predicting later failure were jugular refill, anal tone (but often not recorded, so must interpret whether this is a good predictor with caution), wounds (same caveat as anal tone), heart rate, and heart rate adjusted. The adjusted heart rate is an adjustment that accounts for how long the horse has been in the vet check before the heart rate was taken to compensate for additional recovery time. The heart rate for finishers was 52+/-5 versus 54+/-4 for nonfinishers. When looking at heart rate adjusted (HR + #min waited based on PNR and in-time) the rate for finishers was 57+/-8 versus 61 +/-7 for nonfinishers. As you can see, even though HR (especially adjusted heart rate) is a good predictor, there is significant overlap between the categories of finisher and nonfinisher……It would be difficult to establish a cut off time based on heart rate that captured horses likely to get pulled, while excluding horses likely to finish.
Bloodwork
In 2013 (as in 2012) PCV, sodium were poor predictors for reasons already discussed. The next worst are BUN and bicarb (bicarb could possibly be more significant in a larger study).
Chloride and TP, which were found to be good indicators in the 2012 study were also good indicators in 2013. This is very exciting because it means that this finding was repeatable. Remember that low chloride (chloride is lost in the sweat in large quantities in endurance events…) is associated with failure (getting pulled). However, dehydration concentrates the blood….which can make the chloride actually look higher than it really is in an individual horse. So, they came up with a way to adjust the chloride value based on the total protein (TP). TP increases in dehydration because the blood is becoming more concentrated. So, by combining Cl and TP (something they called a “corrected chloride”) they came up with a number that was very well correlated with whether a horse would go on to be pulled later on in the ride.
How good was it in predicting who would go on to be pulled at some later point? If a horse had a corrected Cl value of less than 84…there was a 70% chance of being eliminated from the ride at some point. If the corrected Cl was less than 80, there was a EIGHTY percent chance of being eliminated at some future point.
For comparison….looking at heart rate, which was the most significant PE parameter in predicting success, the best that we can do is predict about 40-45% elimination.
If someone told me at Robinson Flat before I went down into the canyons that there was an 80% chance that I would be pulled from the ride at some point, I probably would elect to ride another day. Considering that with heart rate the best they can do is somewhere in the 40’s….I would probably take my chances in the canyons and beyond.
Lameness versus Metabolic
The lameness and metabolic pulls were all pooled in the statistics above. One thing that was mentioned is that while we code pulls as “lame”, “metabolic”, “rider option” and that these are good from a practical standpoint, when the data is actually analyzed, there are no “hard and fast” rules that a lameness pull couldn’t actually be better classified as a metabolic pull and vice versa. Some other interesting facts:
- In both years, when rider option vet card/blood work were evaluated, these pulls often fit better into the metabolic pull group from a statistical standpoint.
- I *think* he mentioned that overtime pulls also resembled metabolic pulls – but I don’t have it written in my notes so I could be remembering incorrectly.
So what now?
Unlike PE parameters the blood work has been done blinded to the rider up to this point. While we (as the rider) might change how we manage our horses between now and the next check depending on what the vet says at this check (which may be one reason the PE parameters are not as significant as they could be….), since we don’t know what the blood work says, then we can’t respond to it. There was a lot of discussion of how we might use the information. Create statistical cut off values to generate chances of failure and present as FYI to the rider? Actually pull horses based on the numbers (this was specifically cited as something that is NOT being considered right now)?
The biggest danger seems to be using the numbers as a diagnostic for “treatment” – ie the rider supplementing electrolytes based the serum results. I think as riders we have a tendency to want to “do something” when we see numbers or scores – and I think while we are at the point in our knowledge that while perhaps they can say that this is or isn’t that horse’s day – attempting to correct a low chloride or respond in a specific way to manage the chloride/electrolytes based on the numbers during a ride was called “dangerous”. For example, chloride may actually be a flag/marker for something else and NOT something that can be fixed and have everything be OK.
We know dehydrated horses get into trouble. We know that chloride and TP can indicate something is going on. But there’s a lot of mystery in the black box right now.
That’s it for tonight folks. I’m loopy and exhausted so I’ll finish up the rest of the research presented in the next post. That’s it for the Tevis study, so we will move onto some other recent studies next (good news for those of you sick of hearing about Tevis already and are dreading the invariable Tevis buzz in the next 5 months).
Edit: Gail asked the question in the comments that reminded me of 2 really important points I had meant to cover in this post, so please be sure to scroll down and take a look at my response regarding why the statistical tests are only useful at large rides with an increased pull rate, and why the PE exam is still very relevant (may not be as predictive – but it’s an important assessment of where the horse is RIGHT NOW). (on a side note, 8 hours of sleep is magical and I’m feeling SO MUCH BETTER this morning!!!! Yeah!).
Really interesting research. Thanks for writing it up. I’m curious about how the findings will affect the future of vet checks. If it is true that a lot of the things that are currently part of the vet check are either too subjective or just not good predictors of completion, it seems like it would make sense to focus on things that are more objective/better predictors. But it sounds like more research is needed. I wonder if it would be helpful to start a voluntary collection of data from rides all over the country? I know that introduces selection bias, but it seems to me that just doing the research at Tevis may not give a complete picture, especially in areas of the country where humidity is likely to be more of a factor.
Thanks for reminding me of 2 points that I wanted to make. First, these sorts of statistical approaches to rides only work when the rides are large and the pull rate is higher than usual. Tevis has a 50% pull rate. Most rides have a pull rate that is a small fraction of this. The reason why there has to be a significant “risk” for both outcomes (failure and success) has to do with the positive and negative predictive value (search for a post named “testing the test” for more details on this concept) which basically says that if there is a very very small chance of failure, then there is an increased chance of a false positive using a test – meaning that it is MORE likely that we are going to pull a horse that actually would have gone on to complete the ride.
This means that there are actually very few rides in the country that doing this sort of analysis on blood work is suitable for. And yes, there was discussion that parameters/values of chloride would have to be “validated” in different parts of the country, climates etc. The actual parameters (Cl, TP) are still going to be the big players (they are hte ones that are showing up in research done outside this country in Spain etc too), but the actual numbers that represent “80% chance of failure” may change.
Second point is about the physical exam – while most of the parameters won’t tell you your chances of getting pulled later on – that isn’t to say that they PE is worthless – it’s VERY good about telling us how the horse is doing RIGHT NOW. And because we are able to respond to the PE scores we receive during the ride – we (as riders) may be altering the outcome and making the PE a poorer predictor than it otherwise would be. For example, if you get a poor score for a tack gall – and you change the tack and “fix” the problem – as opposed to just continuing on exactly as you had before the check – you are changing the predictive value of the PE parameter. Obviously the PE is giving the rider useful information that is “actionable” in some ways RIGHT now (even if it isn’t predictive). The unknown thing about the blood values is how they are going to be the most useful. We were cautioned against looking at them as “actionable” – but perhaps there’s a way to integrate them based on their predictive value? (which again remember would only work at larger rides with a larger pull % because otherwise their predictive value decreases, and would probably have to be “validated” on that ride population/region).
Does this make sense?
Absolutely makes sense:) Living so close to OD rides and hearing how difficult the terrain is (and having ridden some of it) makes me forget that not all rides are super challenging:) And I didn’t mean to suggest we shouldn’t do the physical exams, just that it seems like we should make sure we are focusing on things that are meaningful. Given all the variables, I still think knowing your horse as best as you can is probably the best way to have successful ride.
So participation in the 2012 study was mandatory? Did they say what caused the move to optional in 2013? At what point did the rider have to decide whether to participate? The coorelation with RO and OT _is_ really interesting.
We’re looking at prediction of completion only here, correct? Not prediction of vet score at finish?
I am embarrassed to ask this, given my future-epi-student status and all, but can you explain how the heart rates are predictive given the amount of overlap?
If someone told me at Robinson Flat before I went down into the canyons that there was an 80% chance that I would be pulled from the ride at some point, I probably would elect to ride another day. Considering that with heart rate the best they can do is somewhere in the 40′s….I would probably take my chances in the canyons and beyond.
When you put it like that…that is REALLY striking.
…when rider option vet card/blood work were evaluated, these pulls often fit better into the metabolic pull group from a statistical standpoint.
Better than they fit into the lameness group? Or better than they fit into a distinct group of their own?
Thank you for digesting this for us and writing it up — it is fascinating stuff!
I’m not sure what moved the participation from mandatory to voluntary.
That was the point of the overlap – it is predictive from the standpoint that there was a p value associated with heart rate that was significant….but when you look at the actual heart rates with the range…there is no way you could establish a cut off. Something that can predictive and at the same time not practically useful for establishing cut offs in a test, which is what I think is going on with heart rate.
Yep – R/O’s fit the profile of a metabolic pull rather than a lameness point. Remember that he also said there is significant overlap between what’s classified as a “lamness” pull and a “metabolic” pull – so it’s probably best to take those “diagnosis” with a grain of salt. lameness pulls could be do to an underlying metabolic issue etc. But in general, those horses that riders are choosing to pull are mostly fitting the profile of a metabolically pulled horse. (and from personal ancedotal experience – that’s what my RO was – a metabolic problem).
My thought on muscle tone is that it’s best treated as pass/fail. An F/fail on muscle tone means the horse is actually tying up right now, and anything else just means “well it hasn’t tied up yet and we don’t know if it will” – yeah? It’s something they have to check, but an A/B/C/pass muscle tone doesn’t mean anything for the future.
I’d guess that rider pushback is what changed the blood draws from mandatory to optional. I mean, there’s a non-trivial percentage of otherwise great horses who FLIP THEIR SHIT about needles. Blood draws have never been required in endurance, so owners of needle-shy horses have just worked around the issue twice a year rather than trying to solve it.
I would love to see the biggest east-coast rides do bloodwork, even if they’re only doing it on 50s. (No other hundred comes close to getting the sheer number of horses as Tevis, and most of the other hundreds are so small that they’d be statistically useless.) But I’d also love to see West get its own horse scale, like SERA has — but West is the region without a regional organization, so!
Agree with you on the needle shy being a likely reason that it became voluntary….but why were people who rider optioned more likely to be not blood tested? Needle shy cant be the only reason that is associated…
Great post, thanks so much!!!
Could the variation of resting HR between individuals (breeds) be a factor in HR being such a poor predictor?
My thought on the breed is that it doesn’t produce as much variation in the data as you would think. On my arab I can canter into a check and meet criteria in a reasonable amount of time. On my standardbred I walked into a check. The other thing to realize is that different breeds might have different rates of success/failure at the ride. And if they do have higher heart rates/adjusted heart rates and ALSO get pulled at a higher rate, then they are following the trend of the larger group of animals.
I speak of “mays” and “possibilities” because I’m not so sure that the breed difference in heart rate coming into a check (again because of management factors such as the speed chosen by the rider coming into the check) really produces significiant differences – that would have to be measured and analyzed to confirm.