About Us
The Research Process
Research Techniques
Seminars
- > About the University
- > Research
- > Faculties
- > Library
- > IT Services
- > News
- > Events Calendar
INFERENCE and NUMBERS
When should you use numbers, or 'quantitative' methods? You need to make this decision on the basis of the type of arguments you intend to develop. Then you can decide whether you need to count anything, a nd if so, what. Numbers have two main uses in research.
Comparison. When comparing two or more different ways of doing things, numbers provide a base line of comparison. To compare case flow rates in different courts, for instance, or to compare the cost of different programs, we can count the days or the dollars, and say one is faster or cheaper than another.
Unless we do have comparable numbers, a single measure is often fairly meaningless. If a court takes a median time of 10 months to dispose of cases, is that fast or slow? If it costs $2.5 million dollars to run, is that expensive? In fact we cannot tell, unless we are comparing it with other similar courts, which collect their data in the same way. Be wary of using numbers for numbers' sake, or trying to draw conclusions from them without strict comparisons.
Consider Chief Judge Reg Blanch's critique of the Productivity Commission's Report on Government Services 2002 comparing court performance.
Chris Merritt, 'Judge Attacks Court Comparisons' (p 56) and 'NSW Supreme Court Still Nation's Slowest' (p 57), Australian Financial Review, 4 February 2002.
Extrapolation. Because they are capable of being manipulated through various mathematical functions, numbers can also be useful to extrapolate from one or a small number of cases to a larger number. If we know that one court costs $2.5 million to run, then the cost of four comparable courts can be expected to be $10 million. Multiplication, like this, is one of the simplest mathematical functions we can apply to numbers.
The more sophisticated statistical techniques that are commonly used in research are a variation of this sort of extrapolation. Typically, statistical techniques allow us to measure the reliability of our extrapolation from a small group (the one we can actually survey or observe) to the whole population. This measure is expressed in terms of the probability that the results from our sample are the same as the results we would get from the population. It is a measure of the confidence with which we can draw inferences from one group to a larger group. (See Sampling, below.)
Occasionally numbers may be useful in measuring opinions or quality. If that is what you are thinking of using them for, remember that they are still only useful for comparisons or for extrapolation. If the difference between the opinions of two groups is important, or if you need to know how likely it is that your sample's opinions represent that of a whole group, then you may need to use numbers. You should also reflect on what an opinion is and what it tells you about your topic.
In using statistics or other forms of measurement or calculation, it is important to
- be sure you are measuring what you are talking about (validity)
- be sure your different measures are comparable and consistent (reliability)
- be clear how your numbers fit into your argument
- make sure you understand the statistical or mathematical principles of your methods (by reading), and then, if you need to use more sophisticated techniques, get advice on statistics from qualified people.
Sampling. It is rarely possible to count or measure every instance. This is usually only possible where you use existing statistics, such as the population Census or the broad statistics included in annual reports. These have often been collected for a purpose other than the specific topic of your thesis, and so may not tell you what you want to know.
The commonest question about sampling is, 'how big should my sample be?' This is a very difficult question to answer. The answer is often likely to be 'how much time have you got?' It is also not a very useful question. The most important question to consider about sampling is, rather, 'how is my sample different from the rest of the population?'
Sometimes a smaller sample may be a better sample. For instance, if you want to know something about all the legal practitioners in Australia, and for argument's sake there are 100,000 of them, you could get a large sample by sending a questionnaire to all of them. If 5% responded, you would have a sample of 5,000. But those would be the only 5% in Australia who answered your questionnaire, and they would be a pretty atypical bunch. Particularly if there were any reason to expect that propensity to return a questionnaire had anything to do with what you were surveying (even to the extent of being interested in your topic), then you would expect that you had a biased sample.
On the other hand, if, by making phone contact and chasing people up you could get at least 50% to answer your questions, you would have a less skewed sample. In this case, if you selected a representative or random sample of 500 lawyers, and had 250 responses, you may actually have more confidence in your sample being representative of all the lawyers in Australia than if you had a self selected sample of 5,000.
As can be seen in this example, statistical questions don't always have mathematical answers. This is another reason for understanding the principles behind your quantitative techniques before starting to do calculations that you may not understand so well.
A simple and common sense approach to sampling can be found in Yoland Wadsworth's Do It Yourself Social Research. Michael Scriven's Reasoning has a good chapter on the relationship between generalisation to a population from a sample and extrapolation from a group to an individual. For a more technical overview of measurement in social research, see chapter 3 of Social Research by Sotirios Sarantakos.
Frances Clegg's Simple Statistics (Cambridge UP, 1982) is not included in the readings, but is a useful little introduction to some of the terms and concepts.
For more assistance in using statistics, you may consult the University's Statistical Consulting Service.
- Wadsworth, Yoland. Do It Yourself Social Research. 2nd ed. St Leonards, NSW: Allen & Unwin, 1997. pages 53-54.
- Scriven, Michael. Reasoning. NY: McGraw Hill, 1976. Chapter 7.
- Sarantakos, Sotirios. Social Research. South Yarra, Vic: Macmillan Educational, 1998. Chapter 3.