Tuesday, January 15, 2019

We should make academic knowledge easier

If academic knowledge were simpler to understand and use, more people would understand more, misleading misunderstandings should be less prevalent, the education industry would be cheaper and more efficient, and humanity would make faster and better progress. I am convinced this is an idea with enormous potential, but it does not seem to be on anyone's agenda, and there are very strong vested interests opposing it.

Human civilization depends on knowledge. Lots of it, ranging from how to use Pythagoras's theorem to produce a right angle to the science behind mobile phones and GPS, from the idea that germs cause disease to the science behind modern medicine, from stories and ideas about to produce them to the theories behind voice to text software. There are lots of different types of knowledge, and the boundaries of what counts as knowledge are a bit fuzzy. I'm talking here about knowledge in people's heads, not the knowledge in databases and AI algorithms - although the implications of these for what people need to store in their heads is another, vital, and fascinating, story.

Some knowledge is easy and we pick it up naturally as we grow up. But some of it is complicated - it's difficult to learn and use: this is the rather fuzzily defined "academic" knowledge that I'm concerned with here. Two massive, interlinked, industries have evolved to cope with these difficulties: education which disseminates the knowledge, and what, in the absence of a suitable word, I'll call the knowledge production industry or KPI. (In universities knowledge production is called research, but from my point of view this term is too restrictive because it seems to imply the search for the "truth" by modern academics, whereas I need a term which also covers the work of Pythagoras and people trying to devise ways of making driverless cars.)

The KPI - scientists, researchers, and innovators both now and throughout history - make discoveries or invent theories or better ways of dealing with the world, and the results of their labours are then passed on to a wider audience by the education industry - schools, colleges, universities, textbooks, etc.  The education system gets a lot of analysis and criticism: better ways of teaching and learning are proposed and tested, and inequalities of access and the ineffectiveness of a lot of the education system are bemoaned ad nauseam, and so on. But the knowledge itself is seen as given, fixed, handed down from the experts, and the job of education is to pass it on to students and the wider public in the most efficient possible way.

My suggestion here is that there are often good reasons to change the knowledge itself to make it simpler or  more appropriate for the audience. An important  KPI (key performance indicator) for the KPI (knowledge production industry) is the simplicity of the product.

This idea stems from a number of sources, some of which I'll come on to in a minute, but first a little thought experiment. Imagine that a bit of knowledge could be made simpler by a factor of 50%, so that, for example, the time needed to learn it, or to use it, was halved, or that it led to about 50% fewer errors and misconceptions in implementation. Imagine this applies to all knowledge taught in universities and similar institutions. Then students would learn about 50% more, or they would understand about 50% better, or have about 50% of their time free to do something else. Leading edge researchers would arrive at the leading edge in the half the time they take at the moment, giving them more time to advance their subject. If such simplifications could be made across the whole spectrum of knowledge, this would represent an enormous step forward for humanity.

You might think that the innovators and researchers of the KPI would have honed their wares carefully to make them as simple as possible, so a 50% improvement is simply impossible. But you'd be wrong. Very wrong. Except at the leading edge there is absolutely no tradition in the academic world of trying to make things simpler. Simplicity is for simple people, not academics who are clever people. I've had a paper rejected by an academic journal because it was too simple: it needed to be more complicated to appear more profound. Teaching and learning methods are tweaked to make them easier for learners, but the knowledge itself is considered sacrosanct: the experts have decreed how it is, and that's it.

There are exceptions: areas where simplicity is a prized quality of knowledge. One interesting example is the leading edge of one of the most complicated areas of human knowledge: the physics of things like quantum mechanics and cosmology. I've just been watching an interview of the physicist Roger Penrose who was recounting his difficulties with lectures at Cambridge University: they were too complicated to understand so he had to invent simpler ways of looking at the issues. Einstein is supposed to have said that everything should be made as simple as possible, but not simpler. I also came across similar sentiments by two other Nobel prize winners, Paul Dirac and Murray Gell-Mann, and yet another Nobel winner, Richard Feynman, invented a type of diagram -(subsequently called Feynman Diagrams) which gives "a simple visualization of what would otherwise be an arcane and abstract formula" (Wikipedia). Where things are really difficult, simplicity is essential. But behind the pioneers of the discipline, the normal practice is to accept what the gurus have produced.

The history of science and the growth of knowledge in general are punctuated by occasional revolutions that often lead to far simpler ways of looking at things. The invention of the alphabet made record keeping far easier and more flexible so that all sorts of stories could have a wider audience, and the replacement of Roman numerals by the current system (2019 instead of MMIX) did a similar job for arithmetic. The ideas introduced by Galileo and Newton provided a way of understanding and predicting how things move which can be summarised in a few simple equations and covers everything, both on earth and in the heavens. This would probably not have been considered simple by contemporaries of Galileo and Newton, or many present day students, but the equations are staggeringly simple when you consider what they achieved. Similarly, Charles Darwin's theory of evolution by natural selection provides a ridiculously simple explanation of the evolution of life on earth.

But what about the detailed, mundane stuff that students spend their time learning? Quadratic equations and statistics, chemistry and the methodology of qualitative research, medicine and epistemology? Are there opportunities for simplification here?

My contention is that there are, and the fact that are almost never taken is a massive lost opportunity. There are two important differences between the situations of the leaders and followers in a discipline. The first is that the leaders will have a really good understanding of all the stuff leading up to their innovation - the mathematics, other results in the field, the meaning of the jargon, and so on. The followers are, inevitably,  not going to have such a thorough understanding of the background (they've got better things to do with their time). The second is that the motivations are likely to be different. The followers will want to fit new ideas into the mosaic of other things they know and the current concerns of their lives with as little effort as possible; the leaders, on the other hand, are likely to have a burning desire to progress their discipline in the direction they want to take it. These two factors mean that the best perspective for the followers may not be the same as for the leaders.

But is this possible? Are there alternative, simpler, or more appropriate, perspectives in many branches of knowledge? Well, yes, there are: difficult ideas have often spawned popular versions, or, as cynics would say, they have been dumbed down for the masses. But pop science is not usually serious science: if you want to use the ideas for real, or make breakthroughs yourself, the dumbed down, popular version will not do: you need the original ideas produced by the leaders, the experts themselves.

This is not what I am talking about here. What I am suggesting is possibility of producing a simpler more appropriate version for the followers, but one that is as useful and powerful as the original expertise produced by the experts. Or, possibly, more useful and more powerful.

I used to be a teacher in a university, several colleges and on short courses for business. As a teacher you try to explain your material as clearly as possible. But often, perhaps usually, I found myself thinking of alternative ideas which I thought were more appropriate. And I've been doing this for 40 years, publishing the occasional article on what I came up with (the first such article was published in 1978: there is a list of a few more here).

The area I thought about in most detail was statistics. There are three key innovations I would like to see promoted here. The first is computer simulation methods: instead of working out some complicated maths for lots of specific situations, you just do some simulation experiments on a computer so that you can, literally, see the answer and where it comes from (e.g. Bootstrap resampling ...). The second is jargon, which needs changing where it is misleading. The worst offender is the word "significant". This has a statistical meaning, and a meaning in everyday language which is completely different. This leads to massive, and entirely predictable, and avoidable, problems. The third is to focus on ideas that are helpful as opposed to ideas which fit statistical orthodoxy - see for example Simple methods for estimating confidence levels ... .

Other areas I pondered include research methods as taught in universities (a lot of the jargon is best ignored: see Brief notes on research methods and How to make research useful and trustworthy), decision analysis (The Pros and Cons of Using Pros and Cons for Multi-Criteria Evaluation and Decision Making), statistical quality control, mathematical notation in general, Bayes' theorem in statistics (see pages 18-22 of this article), and the maths of constant rates of growth or decline (traditionally dealt with by exponential functions, calculus and logarithms but this is quite unnecessary).

Did I act on these ideas and teach the simpler versions that I felt were more appropriate? Sometimes I did, but usually I didn't. I was paid to teach the standard story, and didn't feel I could go out on a limb and teach my own version - which was usually untested and might not work. That's what the students and the organisations I was working for expected. And, besides, the system has an inertia that makes it difficult to change just one bit. The term "significant", for example, might be, in my view and the view of many others, awful jargon describing an awful concept which promotes confusion and discourages useful analysis, but it is very widely used in the research literature so people do need to know what it means.

There were exceptions where I did follow my better judgment. Computer simulation methods in statistics are widely used (but usually only where other approaches fail) so, on some courses, I did use these. And sometimes, as with research methods, the problem was that a lot of the standard material was just a waste of time and was best ignored so that we could focus on things that mattered. But even here, by not explaining the t-test, or emphasising the distinction between qualitative and quantitative methods, I was failing to meet the expectations of many colleagues and students.

But surely, you're probably still thinking, if there really is such an enormous untapped potential, people would be tapping into it already? Part of the reason why they aren't, or are to only a very limited extent, is that the forces which act against change are very powerful and go very deep. I was so deeply enmeshed in the assumptions of academic statistics that an obvious alternative to the concept of significance in statistics (as explained in Simple methods for estimating confidence levels ...) did not occur to me for 30 years after publishing an article critical of the concept, and the statistics journals I submitted my idea to rejected it often with a comment along the lines of "if this was a good idea the gurus of statistics would have thought of it".

As well as the inevitable conservatism of any cognitive framework there are three factors which are peculiar to the knowledge production industry: the peer review system, the lack of a market or responsive feedback system for evaluating ideas and theories, and the desire of the education system to preserve "standards" by keeping knowledge hard. I'll explain the problems with each of these in the next three paragraphs.

The peer review system is the way new academic knowledge is vetted and certified as credible. Articles are submitted to a journal in the appropriate field; the editor then sends it out to two or three peer (usually anonymous) reviewers - often people who have published in the same journal - who make suggestions for improving the article and advise the editor on whether it should be published. The fact that an article has been published in a peer reviewed journal is then taken as evidence of its credibility. This system has come in for a lot of criticism recently (e.g. in Nature): mistakes and inconsistencies are common, but one key issue is that the reviewers are peers: they are in the same discipline and likely to be subject to the same biases and preconceptions. Peer reviewers would seem unlikely to be sympathetic to the idea of fundamentally simplifying a discipline. I think some non-peer review would be a good idea as advocated in this article.

Mobile phones and word processors are relatively easy to use. You don't need a degree or a lot of training to use them. This is because if people couldn't use them, they wouldn't buy them, so manufacturers make sure their products are user-friendly. There are lots of efficient mechanisms (purchase decisions, reviews on the web, etc) for providing manufacturers with feedback to make sure their products are easy to use. The academic knowledge ecosystem lacks most of these feedback mechanisms. If some knowledge is a difficult to master, you need to enrol on a course, or try harder, or give up and accept you're too lazy or not clever enough. What does not happen is the knowledge producer getting a message along lines: "this is too complicated, please simplify or think again."

This is reinforced by the education system which has a strong vested interest in keeping things hard. There is an argument that the purpose of the education system isn't so much to learn things that are useful (everyone knows that a lot of what is learned is immediately forgotten and never used), but to "signal" to potential employers that you are an intelligent and hard-working person (an idea popularised by the book reviewed here). From this perspective difficult knowledge is likely to be better for differentiating the worthy from the less worthy students. The fact that the difficult knowledge may not be much use is beside the point, which is that the less diligent and intelligent should fail to understand it. And of course difficult knowledge enhances the status of teachers and means that they are more obviously necessary than they would be if knowledge were easier. Universities would lose most of their business if knowledge were easy to master: teaching and assessment would be much less necessary.

Knowledge, of course, is not just to make the economy more efficient; it is part of our cultural heritage and what makes life worth living. Arguably, we have a duty to pass on the work of the masters to future generations? OK, some unnecessarily complicated theories may have historical or aesthetic value, but, in general, if there is a simpler, more elegant version, isn't this preferable?

So ... I would like to propose that simplifying knowledge, or making it more appropriate for its purpose, is an idea that should be taken seriously. Otherwise knowledge will evolve by narrowly focused experts adding bits on and making it more and more complex until nobody really understands what it all means, and progress will eventually grind to a halt in an endless sea of technicalities.

This requires a fundamentally new mindset. First we need some serious creative effort devising new ways of looking at things, and then empirical research on what people find useful, but also simple and appealing. Perhaps knowledge should be viewed as art with aesthetic criteria taken seriously? Whatever we are trying to do - discover a theory of everything, cure diseases, prevent suffering or make people happier - simplicity is an important criterion for evaluating the knowledge that will best assist us.

Then we should make faster progress, more people will get to understand better, less time will be wasted on unnecessary complexities, and we should make fewer silly mistakes.

This is just a summary. There is more on this theme in the articles linked to this page

Wednesday, April 20, 2016

Why is the statistical package SPSS so unhelpful?

I've just run a statistical test on SPSS to see if there is a difference between articles in the Guardian and Telegraph in terms of Characteristic X (it doesn't matter what X is for my purposes here). The results are pasted below. The presence of X is coded as 1, and its absence by 0.

The first table shows that a higher proportion of Guardian articles (33.5%) than Telegraph articles (24.1%) had X. The second table addresses the issue of statistical significance: can we be sure that this is not a chance effect that would be unlikely to recur in another sample of articles?

Paper * Code Crosstabulation

% within Paper
% within Paper
% within Paper

Chi-Square Tests

Asymp. Sig. (2-sided)
Exact Sig. (2-sided)
Exact Sig. (1-sided)
Pearson Chi-Square
Continuity Correctionb

Likelihood Ratio
Fisher's Exact Test

N of Valid Cases

a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 24.21.

I decided I would like a two sided significance level, and looked at the second table to find it. Unfortunately there are no fewer than four different answers (0.128, 0.168, 0.122 and 0.145)! Which to choose?

Further study of the table only deepened my confusion. The heading is Chi-Square tests but two of the columns are headed Exact Sig. My understanding is that the chi-square test uses the chi-square distribution which is a well known way of working out approximate probabilities. The exact test works out the equivalent probabilities directly without using the chi-square distribution, so the entries in the Exact test columns are not chi-square results despite the table heading. One of the rows is headed Fisher Exact Test and another Pearson Chi-Square which seems to confirm this. But what can we make of the top right figure (0.083) which is Chi-square according to the table heading, Pearson Chi-Square according to the row heading, and Exact Sig according to the column heading? Help!

OK, I know I should have consulted the Help (it doesn't work on my computer so I can't), or a book on using SPSS, or gone on a course and provided employment for an expert. But I don't think this should be necessary. SPSS should produce clear  tablese A with a little explanation of what the numbers mean. In the present case, as exact probabilities can be computed surely this is all that's needed. With a sensible heading for the table, and a little note on what the probabilities represent.

SPSS should produce clear, consistent tables which present only the relevant information with an explanation in, as far as possible, non-technical language.

But then people might understand the output and the market for courses and experts would be much diminished.

Thursday, May 28, 2015

Six sigma and the Higgs Boson: a convoluted way of expressing unlikeliness

A few years ago IBM asked me to help them calculate "sigma levels" for some of their business processes. Sigma levels are part of the "Six Sigma" approach to  monitoring and improving business quality developed by Motorola in 1986, and since used by numerous consultants right across the world to package well known techniques in order to con money out of gullible businesses.

The name, of course, was an important factor in helping the Six Sigma doctrine to catch on. It is mysterious, with a hint of Greek, both of which suggest powerful, but incomprehensible, maths, for which the help of expensive consultants is obviously needed.

Sigma is the Greek letter "s" which stands for the standard deviation - a statistical measure for the variability of a group of numerical measurements. Sigma levels are a way of relating the number of defects produced by a business process to the variability of the output of the process. The details are irrelevant for my present purposes except in so far as the relationship is complicated, involves an arbitrary input, and in my view is meaningless. (If you know about the statistics of the normal distribution and its relation to the standard deviation you will probably be able to reconstruct part, but only part, of the argument. You should also remember that it is very unlikely that the output measurements will follow the normal distribution.)

The relationship between sigma levels and defect rates can be expressed as a mathematical formula which gives just one sigma level for each percent defective, and vice versa. Some examples are given in the table below which is based on the Wikipedia article on 25 April 2015 - where you will be able to find an explanation of the rationale.

(An Excel formula for converting percent defective to sigma levels is =NORMSINV(100%-pdef)+1.5, and for converting sigma levels to percent defective is =1-NORMDIST(siglev-1.5,0,1,TRUE) where pdef is the percent defective and siglev is the sigma level. The arbitrary input is the number 1.5 in these formulae. So, for example, if you want to know the sigma level corresponding to a percent defective of 5%, simply replace pdef with 5% and put the whole of the first formula including the = sign into a cell in Excel. Excel will probably format the answer as a percentage, so you need to reformat it as an ordinary number. The sigma level you should get is 3.14.)

Sigma level
Percent defective
Defectives per million opportunities

But what, you may wonder, is the point in all this? In mathematics, you normally start with something that is difficult to understand, and then try to find something equivalent which is easier to understand. For example, if we apply Newton's law of gravity to the problem of calculating how far (in meters, ignoring the effect of air resistance) a stone will fall in ten seconds, we get the expression:
Io5 9.8dt
(represents the mathematical symbol for an integral that I can't get into Blogger.)

If you know the appropriate mathematics, you can easily work out that this is equal to 122.5. The original expression is just a complicated way of saying 122.5.

The curious thing about sigma levels is that we are doing just the opposite: going from something that is easy to understand (percent defective) to something that is difficult to understand (sigma levels), and arguably makes little sense anyway.

In defence of sigma levels you might say that defect levels are typically very small, and it is easy to get confused about very small numbers. The numbers 0.0001% and 0.001% may look similar, but one is ten times as big as the other: if the defect in question leads to the death of a patient, for example, the second figure implies ten times as many deaths as the first. Which does matter. But the obvious way round this is to use something like the defectives per million opportunities (DPMO) as in the above table - the comparison then is between 1 defective and 10 defectives. In sigma levels the comparison is between 6.25 and 5.76 - but there is no easy interpretation of this except that the first number is larger than the second implying that first represents a greater unlikelihood than the other. There is no way of seeing that deaths are ten times as likely in the second scenario which the DPMO figures make very clear.

So why sigma levels?  The charitable explanation is that it's the legacy of many years of calculating probabilities by working with sigmas (standard deviations) so that the two concepts become inseparable. Except of course, that for non-statisticians they aren't connected at all: one is obviously meaningful and the other is gibberish.

The less charitable explanation is that it's a plot to mystify the uninitiated and keep them dependent on expensive experts.

Is it stupidity or a deliberate plot? Cock-up or conspiracy? In general I think I favour the cock-up theory, partly because it isn't only the peddlars of the Six Sigma doctrine who are wedded to sigma mystification. The traditional way of expressing quality levels is the capability index cpk - this is another convoluted way of converting something which is obvious into something which is far from obvious. The rot had set in long before Six Sigma.

And it's not just quality control. When the Higgs Boson particle was finally detected by physics researchers at CERN, the announcement was accompanied by a sigma level to express their degree of confidence that the alternative hypothesis that the results were purely a matter of chance could be ruled out:
"...with a statistical significance of five standard deviations (5 sigma) above background expectations. The probability of the background alone fluctuating up by this amount or more is about one in three million" (from the CERN website in April 2015. The sigma level here does not involve the arbitrary input of 1.5 in the Excel formulae above: this should be replaced by 0 to get the CERN results.)

Why bother with the sigma level? The one in three million figure surely expresses it far more simply and far more clearly. 

Friday, January 16, 2015

Two possible futures

I've just had another conversation with my friend, Zoe, who has solved the riddle of travelling backwards through time. She's just returned from the year 2050: her memories of the future are hazy but fascinating.

In fact she's been to not one future but two - it turns out that all the speculation among physicists about multi-verses is spot on - there are billions of universes, each representing a possible future for us, and she's been to two of them. The rules of travel through time, and between universes, mean that she is unable to remember much detail, but one fascinating point from first of the two universes she went to is that the accepted paradigm in fundamental physics is the "God with a sense of humour hypothesis." Apparently this is the only hypothesis which fits all the known facts, in particular the apparent arbitrary oddness of the laws of nature.

About 20 years ago - talking now from the first 2050 future - two principles from physics migrated to mainstream culture with far-reaching effects. The first was the idea of an absolute limit to the complexity of ideas that the human brain could deal with. The second was the principle that exact laws of nature were unobtainable in the sense that they necessarily needed ideas more complex than this limit. Together these yielded a third principle that knowledge should be designed so as to reduce "cognitive strain" as much as possible. This last principle then led to dramatic changes in the framework of human knowledge. Instead blaming children who found their school work too difficult, extensive research was undertaken to reduce the cognitive strain (or to make it easier). Similar efforts were made with more advanced ideas: for example, Schroedinger's equation - the basic equation of quantum physics that describes how things change through time - was transformed to a user-friendly bit of software with a sensible name that even young children could use and understand. The new version was formally equivalent to the original equation, but far more accessible

This change had a number of far reaching effects. Universities stopped providing degree courses for the masses because the content of old-style degree courses was just too easy and commonplace. A lot of it, like Schroedinger's equation, had entered mainstream culture, and some of it was accessed on a just-in-time basis when needed.

Progress at the frontier of most disciplines had accelerated sharply when these changes came through. The fact that the basics were so much easier meant that there were many more people working at the cutting edge, and the fact that they got there quicker meant that there was more time to work on problems. The old idea that experts spend ten years acquiring their expertise was still true, but the amount of useful expertise you could acquire in your ten years was much, much more.

Cancers, heart disease, and unplanned death in general, were largely conquered, and Zoe was impressed with the solution to the problem of over-population that this would cause, but unfortunately she couldn't remember what this solution was. (Infuriatingly, the rules of time travel and universe hopping set by the God with a sense of humour means that Zoe could only remember a few details of this future.)

The second future had much more in common with the present. The school curriculum was virtually unchanged, university degrees now lasted for ten years, cutting edge research was even more dominated than it is now by professional researchers using language and concepts almost completely inaccessible to laypeople. Cancer and heart disease rates had improved but only marginally.

Zoe much preferred the first future. Unfortunately the God with a sense of humour, while allowing her to go and have a look, and absorb some of the atmosphere, blocked details like how the user-friendly version of Schroedinger's question worked, and the nature of the advances that had largely eliminated common diseases. 

Tuesday, August 12, 2014

The cult of the truth

Everyone seems to believe in the truth. By which, of course, they don’t mean the truth in which other, misguided souls believe, but in their truth which is obviously the right one. The devout Christian has a different version from the devout Muslim, and the devout atheist will think they are both mad.

It is not just religious maniacs who believe in the truth. It is deeply embedded in the world view of science, of common sense, and even fields of academic inquiry which see themselves as being hostile to what they perceive as science. The truth rules supreme everywhere, or so it seems.

But what is truth? When we say something is true we usually mean, I think, that it corresponds to reality – the so-called correspondence theory of truth. But what is reality, and how can human ideas “correspond” to it? Surely human ideas are a completely different type of thing from reality, so what sense does the idea of correspondence make? Perhaps what we see as the truth is part of a dream, or part of a way of seeing the world we make up in collaboration with other people – as the social constructivists would have us believe? This latter perspective seems obviously true (whatever that may mean!) to me - but this may just be the dream into which I've been socialized.

However, let’s accept the idea of truth and try to guess where it might have come from? If we accept the theory of evolution by natural selection, the answer is simple: the idea of truth helped our ancestors survive. A belief in the truth about lions and cliffs helped our ancestors avoid being eaten by the former and falling off the latter. The idea of a fixed reality, which we can apprehend and see as the truth, is obviously a very powerful tool for living in the everyday world. People who did not believe in the reality of lions and cliffs would not have survived to pass on their genes.

This implies that the idea of objective reality and the assumption that we can apprehend the truth about it is merely a human convenience. Frogs or intelligent aliens would almost certainly view the world in very different ways; what we see as truth and what they see as truth would, I think, be very different.

Most statements in ordinary languages presuppose the idea of truth. When I say that Sally was at home at 10 pm on 1 August 2014, I mean that this is a true statement about what happened. Further, if Sally is suspected of murdering Billy 50 miles away at 10 pm on 1 August 2014, then if it's true that she was at home then it can't be true that she murdered Billy. She can only be in one place at one time - "obviously". Ideas of truth, and the "objective" reality of objects in time and space, and the fact that one object can only be in one place at one time, are all bound up in our common sense world view. It is almost impossible to talk in ordinary language without assuming the truth of this world view - it is just "obviously" true.

However the concept is truth is often taken far beyond everyday comings and goings of everyday objects. So we might say that it is true that God exists, that all water molecules comprise two hydrogen and one oxygen atoms, that married people are happier than unmarried people, and that the solutions of the equation x2+1=0 are x=+i and x=-i.

The difficulty is that, outside of the realm of everyday experience, the notion of truth is actually rather vague, may be difficult to demonstrate conclusively, and may come with implications that are less than helpful. Short of taking the skeptic to meet God, demonstrating his existence is notoriously difficult.  We can't "see" molecules of water in the way we can see Sally at home, so the truth about water molecules needs to be inferred in other ways. Saying that the married are happier than the unmarried is obviously a statement about averages - there will be exceptions - and it also depends on what is meant by "happier". And mathematical statements are statements about concepts invented by mathematicians: applying the word true is obviously stretching the concept considerably. It is all much less straightforward than the truth that Sally was at home at 10 pm on 1 August 2014.

The idea of truth has a very high status in many circles. Saying you are seeking the truth sounds unquestionably praiseworthy. If you say something is true, then obviously you can't argue with it. Truth is good so we like to apply the concept all over the place. I'll refer to this assumption that the idea of truth, and the inevitably associated idea of an objective reality with solid objects persisting through time, apply to everything, as the cult of the truth. This notion is rather vague in terms of the assumptions about reality that go hand in hand with the idea of truth - but this is inevitable as the idea of truth gets extended further and further from its evolutionary origin. Cults, of course, depend on vagueness for their power, so that the cult's perspective can be adjusted to cater for any discrepancies with experience.

Does the cult of the truth matter? Does it matter that the idea of truth is extended far beyond its original focus? Let's look at some different areas of knowledge.

Some of the conclusions of modern physics contradict the implicit assumptions of the cult of the truth. At very small scales things can be in two places at once, and reality only makes sense in relation to an observation; for observers at high speeds measurements of physical processes are different, and the notion of  things happening at a particular time depends on the motion of the observer. This all does considerable violence to everyday assumptions about reality, but physicists would simply that these are outdated and that their notion of reality is more sophisticated. It seems to me as a non-physicist, that these theories have sabotaged the idea of the truth about an objective reality beyond repair. I am reading Brian Greene's book, The hidden reality, about parallel universes, but I can't take the idea of truth seriously in relation to universes hovering out of reach which we will never, ever, be able to see in any sense. The hypothesis that the book seems to be driving towards is that we are living in a simulated world devised by Albert Einstein whose theory of general relativity seems to underpin everything.

Does this matter? Probably not for physics. The illusion of the quest for the truth about everything is probably necessary to keep physicists motivated. But in the wider sphere it is worrisome if naive and outdated ideas of physics underpin other disciplines.

The idea of truth is best regarded as a psychological convenience - usually necessary, often useful, but occasionally a nuisance. Am I claiming this statement itself is true? Of course not! My argument obviously undermines itself. But I do think it’s a useful perspective.

Beyond the rarefied world of modern physics the cult of the truth does create problems. Perhaps the most serious is that the status of truth (and science and the study of objective reality) undermines important areas which can't easily be incorporated into the cult of the truth. The ultimate aim of many social sciences is to make the world a better place in the future. We might, for example, be interested in making workplaces happier. The idea of truth fits comfortably with the obvious first stage of such a project - to do a survey to find out how happy workers are at the moment, and what their gripes are. The obvious things to do next would be to look at what the workers want, at what they value, and try to design workplaces to fit these requirement. This seems a more important part of the research than the initial survey, but value judgments, and the design of possible futures, do not fit neatly with the cult of the truth. So they are not taken as seriously as they should be. Most of the thought and work goes into studying the past, and the more important issues of working out what people want and how to design a suitable future, tends to get ignored. OK, so the idea of truth could be extended to include these, but only by bending it so that it gets stupid; the cult of the truth tends to deflect our attention from the problem of designing futures.

In fact the situation is even worse than this because the truths studied in many social science tend to be of rather limited scope. So we study how happy people are in particular organizations at particular times. So what? Everyone knows the situation may be very different elsewhere. The truths studied by physicists are assumed to apply everywhere throughout time (although this can be challenged over billions of years or light-years), but the truths of many social sciences are very parochial. The cult of the truth restricts our attention to trivial questions, dismisses the big questions as trivial because the idea of truth does not apply.

There are further unfortunate side effects from taking truth too seriously. If we have one theory which is deemed true, this may be taken to imply that other theories covering the same are assumed to be false. This may be too restrictive: there could be different ways of looking at the same thing, some of which may, perhaps be more aesthetically appealing, or easier to learn about or use. Truth is not the only important criterion. This is particular true of statistical truths, which may sometimes be so fuzzy as to be almost useless.

So, to recap, truth is best treated as a necessary illusion, often, but by no means always, necessary: it should not be taken too seriously outside the realm of statements about the comings and goings of everyday objects. The last sentence is itself close to asserting a truth whose validity it denies: a fully coherent argument here is not possible, but does this matter? Incoherence gives us more flexibility.