FORUMula1.com - F1 Forum

Discuss the sport you love with other motorsport fans

Formula One related discussion.
#199419
For the past 12 months I have been working on a system of evaluating F1 driver performance.

I have worked for many years within a government statistical department gathering data, ensuring its fitness for purpose, publishing output and interpreting the direction of various industries for both UK and EU government.

The F1 model developed provides a statistical reference point for the level of evidence relating to an individual driver’s performance. Unlike human evaluation it will not make rash predictions relating to driver standards unless there is sufficient evidence to support a claim. i.e. it will not predict a Michael Schumacher or Ayrton Senna after only one race. On the opposite side of the coin it will look at all recent performances and give a rating based on a driver’s more recent results and also a record of the peak rating a driver achieves throughout his/her career. Of course the skills necessary may differ from era to era but each driver performs according to the rules and technology that prevail at the time, and the rating is a measure of how he/she performed against his or her peers given the current technology and rules.

For those of you that understand how an ELO rating system works the output from this methodology behaves in a very similar manner with some exceptions, although I cannot stress highly enough that these are not ELO ratings. E.g ELO ratings do not decline due to inactivity on the part of the rated subject but these ratings would reflect such inactivity. See http://en.wikipedia.org/wiki/Elo_rating ... al_details for an explanation of ELO rating methodology.

Formula 1 is my passion and I am content that the level of accuracy I have now achieved is as high as possible given the limited time and resources available to me as a personal hobby.

I am looking for one of four things.

a) Someone who is willing to invest in further refinement in return for a proportion of rights to future outputs from the system.
b) Someone who can advise me on marketing such data and how I might achieve a reasonable return on the time I have invested to date.
c) An established publisher or author in motorsport who would be interesting in publishing my outputs either in their raw form or as a collaboration in book form.
d) An individual or organisation who is interested in purchasing the exclusive rights to any current or future outputs that are generated from the methodology developed.

I do not wish to go into greater detail here regarding the outputs but to give some idea of the quality of those data being produced the following are some examples:

a) In the first year of formula 1, 1950, with no reference to preceding or subsequent GP data. the system evaluated those results achieved by Juan Manuel Fangio as being superior to that achieved by Nino Farina, despite Farina winning the title.
b) By 1958 Stirling Moss was rated as the top driver at 2686, while Mike Hawthorn in winning the title was at 2638. Most experts would of course agree with this evaluation of each driver given the full access to the history of each driver.
c) In 1962 Graham Hill took the World Title and at that point was rated at 2638. Jim Clark on the other hand was already rated as the best driver in the world at 2670 despite having not yet won either of his titles.
d) The same could be said for Alain Prost in 1984. When Niki Lauda won his 3rd World Title with a rating of 2636, Alain Prost was already achieving the current top rating of 2672, again before he had ever won the title. It is interesting to note that the rating being achieved by Lauda at this point in his career was much lower than his peak in the mid 1970s.
e) Lewis Hamilton's rating stood at 2648 at the end of 2008, his championship winning year. At the end of 2009 in a car that was far less competitive his rating remained at 2646, a difference of only 2 points demonstrating the reduction in the competiveness of his car did not deflate his rating.

Now some people may well say that we know all of the above already and many knowledgeable pundits would have said the same as that being identified by the rating methodology in the above cases. However, I would ask you to consider one point, and that is the opinion of experts is just that. Opinion. The above ratings have been calculated and not influenced by opinion and can be said to be truly objective as the same methodology for evaluating performance has been applied to every driver on the grid at the point in time the rating was being calculated. It is the fact that many of the ratings that are calculated coincide with expert opinion despite their objectivity and methodology that strengthens the case for the validity of the output.

In support of providing further evidence and to achieve one of my four objectives highlighted above, I am here making a limted level of output available to potential investors via Ebay in the form of laminated summary sheets at a minimal charge. These are being made available under the heading of ‘Formula 1 – Statistical Summary Sheets’ and are being sold by the seller <!-- Edited for Content -->. I hope you will all understand that I do not wish to make public all my findings here as to do so would destroy the value of my work. These sheets will be subject to copyright

1. They will identify all drivers since the inception of F1 achieving in excess of a 2600 rating and showing their respective peaks and when achieved. 2600 is the mean rating of all drivers ranked 10th in the world throughout the history of F1, while 2700 is the mean rating of drivers ranked at #1.
2. It will also include current performance ratings of drivers in excess of 2600 who are still active in F1 up to including the most recent GP at time of purchase.
3. It will also include more basic data like total # of F1 GPs won, total # of Fastest Laps and Pole Positions for each driver shown.
4. Finally it will show ‘GP years’ which is a crude measure of a driver’s success assuming that the # of GPs per annum had not been subject to the increase we have witnessed over the years. Each output would come with a page of explanatory notes so that data may be examined and interpreted correctly.
5. The world title in F1 is not a perfect barometer of talent and each sheet will be colour coordinated showing the drivers success in world title terms compared with his rating success.

Should anyone wish to discuss any of the four objectives identified above or wish to ask questions with investment in mind I would only be too happy to hear from you at <-- Edited for Content -->. However, please do not expect a quick response as I am currently working very long days and I am unable to check my emails on a daily basis.

Regards

Tim
Last edited by myownalias on 16 May 10, 07:29, edited 2 times in total.Reason: Removed Email address; will pass this post onto Admins for further evaluation.
#200002
How good a barometer of talent do you believe the world title to be? Do you believe it is a perfect indicator with all multiple world title holders being better than single title holders?

There have been 14 multiple world title holders. If you were to rate your top 14 most talented drivers of F1 history how many of them would be the multiple champions.

After a lot of statistical work, I would say that 9 of the 14 are the multiple champions. 4 are single world title holders and 1 never won the title.

For me the drivers who would force their way in to join our golden 14 and thereby displace multiple title holders would be:

Nigel Mansell = Peak Rating 2734 achieved in 1992 putting him 6th on my list.
Damon Hill = Peak Rating 2711 achieved in 1996 putting him 8th on my list.
Stirling Moss = Peak Rating 2707 achieved in 1959 putting him 9th on my list.
Kimi Räikkönen = Peak Rating 2686 achieved in 2008 (no not 2007) and putting him 12th on my list.
Mario Andretti = Peak Rating 2676 achieved in 1978 putting him 14th on my list.

I am not really asking for praise or criticism for drivers who are on my list. Just wondering how many of the top 14 on other peoples lists would equate to the multiple World Title holders and who of those that are not have made it into your list.

Tim
#200005
This sounds interesting but I cant actually find your 14 golden drivers? Be cool to see the ranked list.
#200029
Not an appropriate methodology as it is wholly quantitative. Statistics produced from a system such as this don't tell even close to half of the story. What about someone such as Alain Prost who could analyse a race whilst on the track and know whether he had to go all out, or whether the sensible thing to do to win the war might include losing a battle or two along the way, allowing others to occasionally pass him and still pick up good points, avoiding the expense an accident might cost him? Statistics don't tell you what is going on inside a racer's head, and whilst we won't know that definitively either, there have been clear indicative signs from many drivers over the years that paint a bigger picture than statistics can show.

The whole reason for many debates, typically between the variety of Schumacher / Senna / Clark / Fangio, is because of these (I won't say intangible elements, because they are tangible, just not conclusive) completely valid factors that statistics forget.

Two of my favourite drivers are Schumacher and Senna, one could make a valid claim for either being the best, however the reality is that there is no overall best ever, Anyone who claims that there is simply doesn't understand statistical analysis (as opposed to simply the production of statistics). There are far too many variables to produce any realistically meaningful and reliable 'statistics' on this. For every ten variables used to produce a mathematical model on this subject, I could point out twenty that have been missed out that would invalidate the findings.

It's the analysis (i.e. the qualitiative analysis, the debates, the rebuttals etc.) that is the most important when discussing F1 champions. Quantitative analysis tells you that a phenomena exists for some reason, qualitative analysis tells you why that phenomena exists. If you use set boundaries in a study of this nature, you therefore limit the remit of the qualitative analysis that follows.

Again, to repeat, and I'm sorry as it sems like you've put a lot of work into it, a study of this nature is utterly useless and invalid. Harsh but true. Sorry... :(
#200030
Oh, I should also point out following on from the above that the results this model would produce would not be 'objective' as stated. They would be classed as 'dependent' i.e. the results produced and their objectivity are dependent on the data input. The objectivity can be skewed depending on the variables and parameters used, and those left out / not defined.
#200076
Thanks Zurich

At least you responded as I was wondering why everyone was ignoring me.

I was trying to put together a kind of sporting rating system that is used in other sports like tennis or chess.

Of course what you have said is not what I wanted to hear and I feel more than a little tearful as a result :( but I am sure you are right.

There are numerous lists and books available claiming to rate the top F1 drivers, e.g. the recent book by Alan Henry. Almost all these are based on opinion. Of course opinion, consciously or subconsciously also rates the various elements of a driver that are important to that individual in his or her evaluation. It’s just that this process is never formalised and the output not quantitative. Alan Henry quite openly discusses this process in his book and the agonising decisions taken as to where to place Clark and Moss.

It was not my intention to say that the output I have generated was an absolute right or wrong answer to this difficult question. It is just one that has been evaluated in a methodical way. No better but no worse than a personal opinion, just a measurable one.

In the same way the lists for tennis or chess provide a rating methodology and an output but no one, and certainly not in tennis, would claim these to be the definitive answer.

When I spoke about the objective nature in my original posting I did not mean they had an absolute answer. Nothing will ever do that. I simply meant that every driver had been treated in the same way and by doing this I had removed any prejudice I may have towards or against certain drivers from the process. You may say that I could be selective on what input data were used to skew the output towards a certain type of driver and that would be a fair statement. All I can say is that I didn’t.

I agree absolutely with your comment ‘Quantitative analysis tells you that a phenomena exists for some reason, qualitative analysis tells you why that phenomena exists’. It was not my intention to say why any phenomena may exist or even to identify what those phenomena might be, to be honest I wouldn’t have a clue. Only to analyse the effect they may have on performance. I’m sure you would agree that Lewis Hamilton and Jenson Button have a hugely differing driving style. It is not the objective of what I am doing to measure this. It only ‘cares’ which one crosses the line first

What I don’t understand and why I feel upset and emotional about this :( is why my opinion is worth nothing from a point of view of discussion (and I am not saying that you said this). As you quite rightly identify there are many threads discussing the Schumacher / Senna / Clark / Fangio debate and by and large people have their say and their opinions are respected. It seems to me because I quantify my opinions they are then dismissed. Why is this?

The above is a serious question as it is something that has plagued my life. If someone jumps up and says ‘I think Schumacher was the dominant driver of all time’, people say ‘quite possibly’. If I jump up and say ‘I think Schumcher was the most dominant driver of all time because if we compare the probability of him crossing the line ahead of his team mate in any individual race we can be 95% certain that he will outscore him on 70% of occasions. We can then quantify the performance of those team mates by comparing their performance in differing teams and with differing team mates throughout their career and thereby derive an overall indicator for those individuals. Then by using a multi iterative process methodology derive an overall rating for Schumacher as well as the team mates’, people say you can’t do that as it’s not a valid methodology. Why is the first person’s methodology of a conscious or subconscious evaluation and non quantitative output deemed worthy of discussion while mine deemed to be unworthy. Surely they should have the same validity?

Sorry for feeling so upset about this I just don’t understand why people are listened to, based on opinion, while my opinion is so often considered worthless because of its quantitive nature.

Again thanks for your time and trouble. Keep in touch

Regards

Tim :(:(:(
Last edited by Wurnos on 19 May 10, 09:19, edited 2 times in total.
#200077
Oh yes sorry Racechick

The top 14 would be

Michael Schumacher 2779 (2004)
Jim Clark 2778 (1965)
Juan Manuel Fangio 2764 (1957)
Alberto Ascari 2748 (1953)
Ayrton Senna 2739 (1991)
Nigel Mansell 2734 (1992)
Mika Häkkinen 2718 (2000)
Damon Hill 2711 (1996)
Stirling Moss 2707 (1959)
Jackie Stewart 2704 (1972)
Alain Prost 2696 (1993)
Kimi Räikkönen 2686 (2008)
Niki Lauda 2684 (1976)
Mario Andretti 2676 (1978)

The bracketed figures are the year they achieved their peak rating.

As stated above this is not a definitive answer to this question, just a methodolgy that uses a quantative approach to drive my individual opinion.

Thanks for your interest.

Apprieciated.

Tim :)
#200083
Pardon me if I'm cynical, but I can't see how you could come up with such a rating system without putting so much subjective choice into the model that you could make it say whatever you wanted it to say (within reason). Also, are there any error bars or statements of uncertainty concerning the final points/ranking? If so, how are they estimated? And how wide are they?
#200090
Wurnos,

I re-read my original post and realised that I perhaps came across as a bit too harsh in what I was saying. I should say that I agree that mathematical, quantitative modeling could be entirely appropriate for some types of sport where the variables aren't particularly complex. If you take a sport such as golf or tennis for example - sure there are adances in equipment technology, training techniques, strength and depth of field etc. howeer to a large extent, much of what you could produce would be valid (within limits), as the general context remains the same - environment, scoring systems etc.

With F1, there are so so many problems that a mathematical model literally cannot ever pick up on or take into account. I'll just list a couple without going into to much detail.

1. The issue I mentioned with Prost. If you look at his post race interview from Monaco 1984, he was asked about the rate Senna was catching him at, and what he would di if Senna was right on his tail. He said that he could have gone faster (I don't doubt that), but opted to ensure he stayed on the track by slowing. He said that if Senna had caught up he would simply have let him past to avoid a potentially race ending battle. He would rather have collected the valuable points for the championship that in normal circumstances he would be fighting for, but Senna would not. A mathematical model would nt pick up on the fact that potentially Prost could have gone faster and could have held off Senna, but chose not to. So the question becomes what is the true ability level? First or second? Of course he won anyway, but had the race been one lap longer before being red-flagged? Impossible to tell.

2. Team orders. Of course now illegal (cynical chuckle), however over the years a number of teams adopted the team tactic of having a strong and preferred number one at a team who would be supported over and above their team-mate. The shift in resource and effort therefore did not always make team-mates equal despite being in the same team. Statistical analysis does not show this. Everyone cites Schumacher at Ferrari here, however there are other notable but often forgotten similar situations such as Senna at Lotus. Senna vetoed having Warwick as a team mate as he feared he would detract attention from his own challenge (people can laugh at that one all they want - it's true!). There are hundreds of other less well reports over the years similar that would not be recognised by this type of analysis. Other teams have in the past went the other way - being so careful to be utterly impartial in maintaining equality between team-mates. With a massive budget this is fine, as both will have an excellent chance. If a budget is not so great though, it is entirely arguable that both drivers might suffer as a result of this type of policy, where one might do markedly better were more resources put their way.

3. Team environment. Some of the best examples of team environment producing notably different results for the same driver that could be cited would be ;

Kimi - ferrari / McLaren
Alonso - Renault / McLaren
Montoya - Williams / McLaren
Frenzen - Sauber / Williams / Jordan
Mansell - Willams / Ferrari / McLaren
Prost - McLaren / Ferrari / Williams
Irvine - Ferrari - and this is even more interesting as the dynamics over one season alone changed several times!

4. Particular car suiting particular driving style. It's widely acknowledged that a driver can have an off year if the way the car responds best to is a way contrary to a particular driving style. i.e. loose rear, understeer or oversteer, sensitiveness etc. Sure you can argue that it detracts from a driver overall if they can't adapt easily, but it equally doesn't let them showcase their talent the way they have the potential to show.

5. Experience of other key personnel apart from the driver. This can have a massive bearing. Zanardi in 1999 is a great example. It's not that widely known that Zanardi's engineering team including his race engineer were utterly fresh to F1, some complete rookies. Where is the logic in giving a proven winner in another formula coming to F1 after a break of 5 years an inexperienced team round him? That was never going to work, and it proved so. Zanardi was a massive talent that went unrealised in F1 through Williams failings in this regard. The one race that Zanardi qualified and raced well in during that season (can't remember off the top of my head) was because the situation reached a boiling point and Zanardi revolted and broke from the norm- a contributing factor to his leaving Williams and F1 again. F1 is not just about the driver - I'd LOVE to see the drivers make their own cars on race weekends, but they don't.

I know I went on a bit there, but hopefully it illustrates the point. I'm sure your model could be useful, and will produce some good results worth debating, I just question their validity and don't see the produced results as being any more valid than other methods used...
#200092
Quotation deleted


Wurnos, I was trying to be reasonable, hence my lengthy replies, however with your last statement on 'how complicated can it be?', you've hit the nail on the head, you're either naive or ignorant on the actual complications that beyond doubt do exist in F1. That's the problem with a quantitative analysis in this regard. I criticised your methodology because you left yourself very open to criticism in the way it was chosen. Even a mixed methodology would have been more valid in this context than a wholly quantitative one, which is just inappropriate however you look at it.
Last edited by zurich_allan on 19 May 10, 11:38, edited 1 time in total.
#200093
My sincere apologies Zurich

The comment you quoted wasn't aimed at you as i felt you were being completely contructive in your criticism even though it might not have been what I wanted to hear.

I'm sorry you felt me to be naive and ignorant but I can understand where u are coming from.

I lost my job as an analyst a couple of years due to depression and I am now working sixteen hours a day 7 days a week across two jobs on minimum wage to try to keep up the payments on my house for my family. As a result of this I am starting to feel the effects of the depression again and I 'm not behaving in a rational way when dealing with criticism.

Grateful if you would delete my quoted text from your previous post as I am quite ashamed of my reaction now and having it available fo everyone to see how stupid I am is distressing me further.

Sorry again if you felt my previous posting was aimed at you.

I think all that you have said is correct.

Take care

Tim
#200094
I've deleted the quote as per your request. Please also don't take what I said regarding naivity or ignorance the wrong way. You clearly come across as an intelligent person (I don't mean that to sound condescending), especially as you clearly have a very sound understanding of both quantitative analysis and advanced mathematical concepts. What you have worked on has obviously taken a lot of time and effort for which you should be commended. I just don't think it is so appropriate for F1, but that is not to say that the concept you've realised is useless. On the contrary, I think it could be very useful if related to the other sports I mentioned, Tennis and Golf, both of which I play, watch and have great interest in. I'm just questioning its use when specifically using it in terms of F1.

I'm sorry to hear about your problems, and I'm sure you can get past them. Not that what I have done in the past would be comparable, and not something I have ever shared openly before though perhaps doing so now would be appropriate, but I went through something similar a about 8 or 9 years or so ago myself. I lost my job due to the company I worked for closing at very short notice. I had a number of financial commitments (mortgage, credit card, loan etc.) that I quickly fell behind on. Instead of dealing with it all in a rational way I buried my head in the sand. I was never officially diagnosed with depression, but I would have a guess that I was suffering from it to some degree. It took until literally a day before my house was going to be repossessed that I finally accepted my situation and started to turn things around. I reached out for help, managed to get enough to stave of the repossession, and gradually managed to pay off every penny I owed over the next three years (still living in that house today!). I turned the negatives of all of this into a positive and decided to go back to university where I first completed my law degree, then a PgC in Research Methodologies, followed by a PhD related to Law and a PgC in Teaching in Higher Education. I managed to get an associate lecturer position while completing my PhD, and am now a full time university lecturer and fellow of the Higher Education Academy. I'm getting married in October and life is wonderful.

What does any of this mean? I firmly believe that we will all have hard times that we must endure in our lives. I also believe that we have it within ourselves to find a way through and come out the other side stronger and happier as a result. I'm so grateful and appreciative for everything I have now, and the opportunities that I'm presented with, as I've experienced the other extremes. I'm sure that when you get through everything that's been thrown at you, you'll find happiness and appreciate it in the very same way that I do.

All the best.
#200148
If you are going to rate the best, their accomplishments as well as their mistakes have to be quantified. Not just what have they brought to their team, but what have their mistakes taken away from the team. That's a difficult feat in itself.

I am also having a hard time seeing how you are going to rate the best drivers in a sport which is such a team effort. Plenty of very talented drivers were wasted on teams that could not compete, either due to money or lack of engineering talent. Plenty of great teams have carried drivers of only decent capability further than their skillset alone would have taken them. Examples of lower formula drivers who dominated coming into F1 and going nowhere are far from uncommon. The team is everything....I believe the team is far more important than the driver. If Hamilton had been put into a Force India...where would he be? Schumacher....same same. The best driver in the best car supported by the best team is a force to be reckoned with, but if you take any of those three factors and decrease it....the other two may still be the best, but they will finish mid pack with no fame or fortune.

F1 isn't a sport where the driver is the end all be all, though it is heading in that direction, with the current trend towards spec racing. In a spec series where the cars are all (on paper) created equal, then perhaps a driver rating system would be possible...but in F1, especially F1 of the past...I think you are spinning your wheels trying to form a system that will show who the best driver is. There is no way to separate the driver from the car and team.
#200167
Thanks Zurich.

Your kind words are much appreciated and it’s nice to know there’s someone else out there who has been to where I am now. It’s great to see there can be light at the end of the tunnel.

Take care and thanks again for everything

Tim
#200172
What your trying to do is impossible, good effort though.

F1 Isn't a spec series, and you can't compare diffrent Era's of cars either.

Fangio was One of the best of his Era.

Senna One of the best of His Time

Schui One of the best best driver of his time.

But swap those drivers around a 50 year old Fangio in a 2004 Ferrari might not produce a world championship.

Think your ELO methord is simlar to the castrol driver rankings.

Which is daft because they are saying Jimmy Johnson who would get pissed on in F1 is better than Lewis Hamilton

See our F1 related articles too!