Evaluating Evidence and Information Sources
By Mario Rautner
In Short: You’ve identified and collected information that may serve as evidence in your investigation. What next? - Learn how to analyse and verify it as well as how to evaluate your information sources to be able to assess the reliability of your findings.
Investigations are only as strong as the evidence you collect for them. Without evidence to support your work, your claims or statements have little substance and will not convince your audience. In some cases, you may even face legal consequences if you are not able to support your assertions with strong evidence.
It is therefore important for investigators to understand the type of evidence they can gather, the reliability and strength of the sources from which evidence is collected and when the combination of evidence is strong enough for the investigation to be turned into a story, a report, a claim, a campaign or made public in any other way, if that’s your final aim.
Developing a hypothesis
A hypothesis is “a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation”. Using a hypothesis will not only give your investigation focus, it will also help guide you through the evidence that you need to collect and to assess information you’ve already gathered. For more clarity, we can use the term “working hypothesis” since it implies a work in progress, which evolves and may change based on the evidence you find (or do not find).
In his co-authored book “Story-Based Inquiry”, Professor Mark Lee Hunter, a founding member of the Global Investigative Journalism Network (GIJN) argues that using hypotheses is the core of the investigative method and that developing a hypothesis gives you something to verify, which means you can assess your evidence against it. This in turn will help you structure and plan your investigation. Such structured research will make it much more likely that you will apply techniques that will lead you to important evidence.
It is also imperative to recognise that your starting hypothesis is not set in stone. It can, and should, change as you discover and interpret new evidence. At times, a starting hypothesis might even look irrational, speculative or ungrounded but as you find more proof, you should be able to adjust and expand it. Not doing so means you risk falling into the trap of wrong theories or conspiracy theories – although even supporters of such theories are able to support their hypotheses with some form of evidence or interpretations of it.
Hypothetical example
You may have received an anonymous tip-off that a tannery (a leather processing plant) is dumping toxic waste into a river in your neighbourhood, killing fish. First, you quickly check a few facts, including applicable regulations for effluent releases and the location of various factories in relation to the river on satellite. You also talk to some residents and one of them reports seeing dead fish in the river recently. At this point you do not know if there is any truth to the information you received, but because there are a number of companies located in the vicinity of the river, you develop an initial hypothesis:
“Someone is illegally releasing chemicals into the river.”
This example is used throughout the guide.
What is proof and what if there is no proof?
An investigator’s main goal is to uncover and report reliable and verifiable facts that can serve as proof. Proof is defined by the Oxford dictionary as “a fact or piece of information that shows that something exists or is true.” Proof can apply to individual pieces of evidence as well as to your overall hypothesis as the sum of the collected evidence. Your goal will always be to collect enough evidence to get as close as possible to proving or disproving a hypothesis. However, it is important to point out that journalists, citizen investigators and other civil society members who investigate have different roles in society compared to scientists or the legal systems. What we may call ‘journalistic approach’ is not the same as ‘scientific approach’ to evidence and investigation. While a working hypothesis directs you in your investigative efforts, your goal is not to prove that your hypothesis is correct at all costs. Instead, your responsibility it to keep an open mind about alternative explanations, even if they do not agree with your own beliefs or convictions.
While your goal is to prove or disprove your hypothesis, keep in mind that it is extremely rare for any investigation to provide 100% proof. More often, the collection of your evidence will result in an approximation of proof where (after looking at all possible explanations) your hypothesis is the most likely explanation of the way events have occurred.
There will be situations where – despite your best efforts – your investigation will reach a dead end. At this point you can either give up on your investigation, change focus by adjusting your hypothesis (even if this challenges your own assumptions) or publish the information you have in a transparent way without making claims for which you do not have any evidence.
Types of evidence and searching for proof
Once you have adopted a working hypothesis, it will guide you through your investigation and evidence collection. There are many different types of evidence that you might come across in the course of your investigation. Evidence may be direct or indirect, obtained through field research or desktop work, and it can be physical or digital. Most evidence falls into more than one of these categories. These differentiations matter when it to comes to interpreting your evidence and ascertaining its strength and reliability. Often the type of evidence is linked to the type of investigation you are carrying out. For instance, field investigation tends to result in physical evidence, such as environmental samples, or in video and photographic evidence.
Direct vs indirect evidence
Direct evidence usually speaks straight to your hypothesis and establishes facts. Examples for this include video, audio and photographic evidence. Eyewitness accounts of events, official documents and records are also considered direct evidence.
Indirect evidence, on the other hand, is usually evidence that supports an assertion but is not strong enough and mostly second-hand and therefore not able to establish facts to the same degree. Examples of this include statements by people who did not observe events directly, statements by spokespersons for others and generally any kind of evidence that requires you to make inferences. An inference is an opinion or conclusion you form based on evidence that does not directly speak to that conclusion. We make inferences every day: for example, when you see a parcel in front of your door you presume it is for you without looking at the direct evidence, the label on the package. This is only strong enough to support a hypothesis if there is an accumulation of indirect evidence and when no other logical conclusion is possible. Most of the time when you come across indirect evidence it will be necessary to carry out further investigations and to verify it with other sources whenever possible.
Example
It is important to keep in mind that even direct evidence may only confirm a specific component of your hypothesis rather than the whole story.
In our example of water pollution, a report from a laboratory that shows toxic synthetic chemicals in a river only establishes the fact of the river having been polluted. It does not provide any information about who caused the pollution, nor does it necessarily show the exact spot on the river where the pollution originally occurred. Examples of indirect evidence with this case study may be an observation by an eyewitness of discoloured water or the discovery of dead fish. None of these examples establishes or disproves whether there is pollution in the river since discolouration and fish mortality can occur from natural causes and a company spokesperson is not able to provide direct evidence.
It is important to keep in mind that what is indirect evidence for one statement or hypothesis, can be direct evidence for another. If a company spokesperson says that “our company does not discharge toxic chemicals into the river,” this can be considered only indirect evidence of the company in fact not causing pollution. It is, however, direct evidence for establishing that the company denies having caused the river pollution.
Human vs physical vs digital evidence
Evidence can originate from a wide range of sources which can be broadly categorised into:
human
physical
digital
Evidence from human sources can come from anyone who is broadly associated with or touched by your investigations, even if just on the periphery. For instance, this can include eyewitnesses, victims, government officials, scientists, spokespeople, journalists or experts. Ultimately, digital and physical evidence are always linked and influenced on some level by human evidence and sources, since it is always originally developed, created or analysed by humans. We’ll discuss this more in the next section on assessing evidence, but for now it is worth keeping in mind that videos and photos are taken by people, that environmental samples are analysed by machines and computers controlled and operated by people and that documents are written and edited by people. People are subjective and have points of views or positions that may affect the evidence they produce.
Physical evidence broadly applies to paper documents, books, magazines, contracts, invoices, environmental samples, some photos and videos and more. Even though our world is increasingly becoming digital, physical evidence is important and is generally more trustworthy than most of its digital counterparts since it cannot be manipulated as easily. Since the digital and physical worlds are increasingly integrating, the boundaries between physical and digital evidence are also blurring. Only twenty years ago, videos and photos would have been considered physical evidence but recent photographic and video evidence is now strictly speaking digital, as it originates from digital cameras. This is one reason why assessing the origin and strength of evidence and the sources that provide it has become so important. A similar trend occurs for instance with documents that are increasingly delivered digitally. Only a few years ago a Freedom of Information request might have yielded hundreds of pages of physical documents. Now it is much more likely that the government’s response to your request includes PDF documents and digital records. In many ways, the conversion to digital media and the associated ease with which information can be altered poses an important challenge for investigators.
Because of the blurring of these boundaries and the rapid move towards digital document provision and storage, we are using a narrow and somewhat fluid definition of digital evidence in the context of this guide. Here, digital evidence only applies to information that is found on the internet. This includes online articles, reports and studies, digital databases, videos and photos that are online but also blogs, social media, discussion forums. Photos and videos straight from the cameras and in their raw formats are considered physical evidence here.
Note however that with some effort nearly everything in the digital world can be manipulated including images and video. The metadata of photographs that you have not taken yourself can be very easily altered and should not be relied on as evidence, unless you can verify and confirm it from its original source(s) and/or by other means such as reverse image searches - using tools like:
and/or, by verifying geolocation and time, etc.
For additional guidance and tips on how to verify images, consult further online resources such as:
this Ijnet list of “9 tools to verify images”, or
this Exposing the Invisible guide on “Metadata Investigations”.
Tip - Deepfakes, videos and photos
The interface between digital and physical evidence is shrinking constantly. This means that videos, photos, PDF documents exist both in the digital and physical realm. Strictly speaking they are largely digital but are often treated as if they were physical. Increasing computational powers of computers, their software and artificial intelligence means that it is becoming harder to identify fakes. Take for instance so called deepfakes, a technology that is able to produce photos and videos that are computer generated but extremely convincing. While at this point it is less likely that you will encounter such fake evidence in your investigations, considering that deepfakes only emerged in 2017 and their use is spreading rapidly, it is feasible that in the near future they can be used by governments, companies or individuals to provide fake evidence of events occurring or not occurring. It has been argued that deepfakes will be capable of “distorting democratic discourse; manipulating elections; eroding trust in institutions; weakening journalism; exacerbating social divisions; undermining public safety; and inflicting hard-to-repair damage on the reputation of prominent individuals, including elected officials and candidates for office” (source: “Is seeing still believing? The deepfake challenge to truth in politics”, by William A. Galston in brookings.edu.)
If you are looking to learn more about deepfakes and the potential of some emerging (and constantly improving) technologies to distort facts and reality, take a look at Tactical Tech’s Deepfake Lab, which provides a series of interactive multi-language examples and explanations of how this is all possible.
Image source: Tactical Tech, ‘Deepfake Lab’ – Glassroom project: https://deepfakelab.theglassroom.org/
Image: DeepFake Future – poster by Tactical Tech’s Glassroom Misinformation project: https://www.theglassroom.org/misinformation/exhibition/
Evidence from desk and field research
There is a lot of evidence that you can gather without leaving your home. Evidence from desktop research includes anything you can uncover with your computer, such as researching scientific or newspaper articles, information from databases, even some video, audio and photographic evidence as well as information gathered from digital maps, satellite imagery and social media. Desktop evidence also includes anything that is emailed to you via Freedom of Information requests or through online conversations.
Field research requires you to leave your house. It can be gathered through a range of activities such as interviews, evidence collection in libraries and archives as well as taking environmental samples. Undercover research often takes place in the field (although some undercover techniques such as using different identities are also widely applied online).
Assessing trustworthiness of scientific papers and journals
Not all the scientific papers you may come across are strong enough in providing evidence and accurate research methods. The strongest research you see published in journals like “Nature” or “Science Magazine” is thoroughly verified through a process called “peer review” whereby other researchers and scientists with the same area of expertise will analyze each new article and study submitted for publication. In many cases, this process requires months, even years, to complete as some research and methodologies (especially lab work) need to be replicated before the results/methods get vetted and receive the “green light” for publication.
There are many journals out there that publish research that hasn’t been peer reviewed yet - not needs to be - or that require a fee from authors to get their work published without much verification. This is called “vanity press”. Some of these journals are far from providing the quality research, which you should trust when seeking evidence.
Beall’s List is a website that comes to our help in such cases. Created by a librarian - Jeffrey Beall - it offers a way to determine to what extent a journal is problematic and potentially predatory by keeping an index of many of the world’s open access journals, as well as by providing a list of criteria by which you can judge a journal’s credibility.
Examples of evidence classifications
The table below has some examples of different kinds of evidence and categorises them according to type (direct, indirect), its sources (human, physical, digital) and the origin of where it has been obtained (desktop, field). As pointed out earlier, the same evidence can be direct or indirect depending on your assertion or hypothesis. Therefore, this table is only relevant within the framework of a hypothesis.
Example
In our hypothetical example the assertion that we test below is: “The Company X is discharging toxic water into the river.”
| Type | Source | Origin | |||||
|---|---|---|---|---|---|---|---|
| Example evidence | Direct | Indirect | Human | Physical | Digital | Desktop | Field |
| An interview with a company representative recorded and seen on the internet stating there is no water pollution by the company | X | X | X | X | |||
| Photos taken by others of dead fish in the river | X | X | X | ||||
| A letter/PDF from a laboratory with the results of the water sample tests showing elevated toxicity | X | X | X | ||||
| Water quality measurements released on the council website showing no elevated water toxicity | X | X | X | X | |||
| Written or recorded statement by a scientist who interprets the environmental impacts of the effluent release | X | X | X | ||||
| A statement by the mayor in a press release about the pollution that she does not believe there is a problem or that it is caused by a company | X | X | X | X | |||
| A resident tells you he has noticed the bad smell of the river water close to company X | X | X | X | ||||
| The CEO admits in an undercover recording that his company has discharged untreated water into the river for years, saying everyone is doing this. | X | X | X | ||||
| Lab tests of water samples of the effluent of the company pipeline show highly toxic discharges | X | X | X | ||||
By the very nature of investigations, direct evidence that proves your overall hypothesis is not easy to obtain and a much likely scenario is that you have to combine many indirect sources or multiple hypotheses to establish proof. If you take each of the pieces of evidence by themselves, they are not very meaningful and do not mean that your hypothesis of river pollution from company X is correct.
It is important to question your evidence all the time and not to reach conclusions that are not clearly supported by your evidence or allow logical alternative interpretations.
Analysing evidence
In order to understand to what extent the evidence you’ve gathered might constitute proof, it is necessary to analyse its strength. Every piece of information or evidence comes with risks and may raise further questions. Therefore, the ability of investigators to question the reliability and strength of the evidence gathered becomes vital. This scepticism towards one’s own opinions and biases is a fundamental characteristic of honest/fair investigators.
Strength & accuracy of sources
When interpreting evidence, it is crucial to understand the reliability and motivations of the source of the information. This means that the same piece of information can be considered strong or weak depending on its origin and the context in which it was provided. It is up to the investigator to interpret the evidence in combination with its origin and source to ascertain its strength.
Some evidence has already undergone at least some sort of review process that makes it generally stronger and more trustworthy.
Scientific articles for instance have generally gone through peer reviews, although one should be wary of pre-prints that have been published before completion of reviews. Special attention needs to be given to the funding sources of such articles as well, since many companies commission research that may turn favourable to their interests.
Reports by respectable media organisations, even if they exist entirely in the digital world, are often reviewed internally by fact checkers. That does not mean you shouldn’t double-check their sources and data, as their information may not always be correct. Checking their affiliation and codes of conduct is also a good way of establishing some trust. For instance, commercial and non-profit media organisations that are part of the International Fact Checking Network or the Global Investigative Journalism Network (GIJN) may have systems in place that result in more accuracy.
The less corrupt and more transparent a government, the more the information they release can be trusted. However, overall national, state and local government information alone should rarely be considered strong enough to contradict your hypothesis on its own.
It is a well known phenomenon that humans often tend to believe what they want to believe and – whether we like it or not – investigators are not immune to such “cognitive bias.”
Note
Even scientists who try to use replicable methodologies for experiments that can be repeated in order to obtain the same results are not objective in all cases. This potential lack of objectivity can influence the outcome of their work. Author Angela Saini wrote that “Scientists who imagine that bias lies in others, not themselves, fail to recognize that to live in the world today is to be drip-fed assumptions and prejudices that guide our thoughts and actions.”
When bias exists for scientists, who generally apply the most objective methods to their work, then this is surely the case for investigators, who generally deal with subjects that do not follow a strict scientific methodology. Understanding the role that investigators themselves play in the outcome of their investigations is just as important. Not doing so carries the risk of including inaccurate information, which can have significant legal consequences for the investigator as well as substantial personal consequences for the people incorrectly accused. This is why, when in doubt, it’s better to presume that you do not have enough proof for your hypothesis.
Cognitive bias
Cognitive bias is a broad term that refers to the way our judgement and decision making is influenced by the context and framing of information and our personal beliefs. Cognitive bias was first defined through psychological research in the 1970s but people have been aware of the phenomenon for hundreds of years.
Cognitive bias can have serious consequences.
An awareness of bias is particularly important for citizen investigators because they often work alone or in a small group, frequently have personal or emotional attachments to the outcome of their investigation and can lack review processes that are common within professional non-profit organisations and the media.
One of the most relevant cognitive biases for investigators is confirmation bias, in which we favour ideas that confirm our existing beliefs and what we think we know. As a result, people might favour specific scientific evidence or other data over contradictory evidence and also come to broad conclusions aligned with one’s bias. Investigators and journalists are not immune from this bias. An experiment that compared journalists with citizens in Germany found that journalists were just as susceptible to confirmation bias.
Another important bias to be aware of is the sunk-cost effect. This is defined as a tendency to continue an endeavour or pursuit once an investment in it has been made. This investment can be monetary but it can also be time and effort. For investigators this is relevant, for instance, if evidence that disproves a hypothesis could easily be ignored especially if it is uncovered after a lot of effort has already been spent on the investigation.
There are many other forms of cognitive bias that can easily influence our ability to carry out investigations objectively.
Availability bias: when people give more credence to solutions and information they remember first, which is often the most recent. This can be a fallacy in investigations, in particular for those that stretch over long periods of time.
Anchoring bias: is nearly the opposite of availability bias. Here people are more likely to believe the initial piece of information that therefore serves as an anchor against which future information and evidence is compared.
The good news is that awareness of one’s biases, the application of specific methods and tools to avoid it and training can all reduce the extent to which cognitive bias affects our decision making and thinking. Some of the techniques that help reduce the risk of bias include:
Seeking second opinions of evidence interpretation by colleagues and other investigators
Reviewing your conclusion after some time has passed and
Making an effort to disprove conclusions and evidence interpretation
Additional techniques can be found online and scientists have even developed computer games such as this one to help people make better decisions and to reduce bias.
During an investigation, you may talk to various people within a company and ask them similar questions.
Example
In our example with water pollution, you might talk to a receptionist, a media spokesperson, a worker and a security guard and ask each of them whether they are aware of water pollution originating at the company.
A company director or spokesperson is likely to have detailed knowledge of the company’s overall activities and plans, but has a lot to lose. This could result in attempts to misdirect or mislead you. A receptionist might be uncomfortable about your question and may not know about this. Security guards and workers may know a lot of what is going on at the company but might face losing their job for talking to you.
In the list below the same statement that the company you are investigating is not polluting the river has been made by a number of different sources. Let’s weigh their strength:
| Evidence/statement | Origin of information | Strength of evidence |
|---|---|---|
| “Company X is not polluting the river” | From an anonymous letter you received | Weak |
| “Company X is not polluting the river” | Interview with company spokesperson | Weak |
| “Company X is not polluting the river” | Twitter message by a local politician | Weak |
| “Company X is not polluting the river” | Conclusion of a study commissioned by the investigated company | Medium |
| “Company X is not polluting the river” | Conclusion of an independent scientific analysis (if you can verify and confirm its independence) | Strong |
| “Company X is not polluting the river” | Local council report with river water test figures | Medium |
| “Company X is not polluting the river” | Report by the local business council | Weak |
| “Company X is not polluting the river” | Interview with a resident | Weak |
Example
There are also many instances where the same sources that provide weak evidence for this negative statement can provide strong for the opposite, positive statement. As shown in the table below, many of the instances of weak evidence when saying there is no pollution become strong evidence when the same sources say there is pollution, showing that the strength of a source is often dependent on the evidence or information it provides rather than being inherently strong or weak all the time. Therefore, understanding the motivation of a source in relation to specific evidence becomes important.
| Evidence/statement | Origin of information | Strength of evidence |
|---|---|---|
| “Company X is polluting the river” | From an anonymous letter you received | Weak |
| “Company X is polluting the river” | Interview with company spokesperson (watch out for possible interests/bias of any company sources) | Medium |
| “Company X is polluting the river” | Twitter message by a local politician | Weak |
| “Company X is polluting the river” | Conclusion of a study commissioned by company | Strong |
| “Company X is polluting the river” | Conclusion of an independent scientific analysis | Strong |
| “Company X is polluting the river” | Local council report with river water test figures | Strong |
| “Company X is polluting the river” | Report by the local business council | Medium |
| “Company X is polluting the river” | Interview with a resident |
Mitigating information risks and evidence verification
There are numerous actions you can take once you realise that your evidence may not provide enough proof to support your hypothesis. For instance, a rule that is often applied in journalism requires at least three sources that are independent from each other to confirm the same information before it can be published.
Example
Let’s use the same type of evidence we have described previously and figure out the information risks an investigator may face, and what can be done to reduce them.
| Example evidence | Risks to your information/evidence | Mitigation actions |
|---|---|---|
| A recorded interview with a company representative seen on the internet stating there is no water pollution by the company | The spokesperson might try to mislead or misdirect investigations | Analyse carefully what the person said. Does this leave other avenues open that are not explicitly denied in the statement? Sometimes spokespeople lie on purpose or because they do not know better. |
| Photos of dead fish in the river | Unless you can be sure of the location and that it is indeed the same river, this evidence does not provide strong support for your hypothesis | Try to identify the exact location where the photo was taken to carry out further field research there. Is this upstream or downstream of the company? How close is it to the leather processing plant? |
| A letter/PDF from a laboratory with the results of the water sample tests showing elevated toxicity | This only shows there is pollution in the river but does not link it to a specific company | Talk to the lab in person to fully understand the results of the tests, where and when the samples were taken. Plan further research to see if the pollution is linked to the company you suspect. |
| Water quality tests of the river from the website of the local council showing no elevated toxicity | Samples may have been taken at a different time or at locations that would not be able to capture pollution caused by company X. | This is only an extract of the actual tests. Request the full tests over a few years including time of sampling, locations, substances tested for, labs used, etc. |
| Written or recorded statement by a scientist who interprets the environmental impacts of the effluent release | Note that this still does not make a link to the company. | Research the scientist further. Do they have links to the industry? What is their reputation? Pay attention to what the scientist says about the timing and location of the tests. Talk to the scientist directly and in person. |
| The mayor says in a press release on the pollution that she does not believe there is a problem or that it is caused by a company | Has a self-interest to make such a statement as it might affect their public support. | Look into any links between the mayor and the company such as campaign contributions. |
| A resident tells you he has noticed the bad smell of the river water close to company X | There are many reasons why water might smell bad | Ask the resident to show you the exact spot where this was observed. Where is this in relation to the plant? Are there any discharge pipes upstream from the location and where do they lead? |
| You were given an undercover recording in which the CEO admits that his company has discharged untreated water into the river for year, saying everyone is doing this. | If published the CEO might say he was misled, or might use legal threats to stop you from using the interview | Show the material to a trustworthy lawyer and get advice how best to proceed. |
Analysing competing hypotheses
One of the best ways to exclude at last some of your bias and tendency to want to support what you believe is to try to gather evidence that disproves your hypothesis. Testing for alternative explanations of your assertion throughout your investigation will help you to be more objective. A technical term for this is an Analysis of Competing Hypothesis (ACH) which is a technique often used by intelligence services. In this process the investigator first generates possible hypotheses, then crates a matrix which is filled with evidence that is then tested for logic and consistency. The hypotheses are then evaluated and ranked.
Note
A fancy term for looking for alternative explanations is a methodology called Analysis of competing hypotheses or ACH, which was originally developed by the CIA in the 1970s. However, unless you are working on hugely complex, serious and long-term investigations, looking for objectively correct alternative interpretations of your evidence using your commons sense is most likely sufficient.
If you are interested to learn more about ACH you can find software, blogs, and articles to start off with.
Example
In our example, two competing hypotheses may be that: “Company X is responsible for polluting the river” and “There are other causes for the pollution.”
C = consistent
I = inconsistent
| Evidence | Company X caused pollution | Something else caused pollution |
|---|---|---|
| You found a pipeline from company X to the river | C | C |
| Photos of dead fish in the river | C | C |
| A letter/PDF from a laboratory with the results of the water sample tests showing elevated toxicity | C | C |
| Water quality tests of the river from the website of the local council showing no elevated toxicity | C | C |
| Written or recorded statement by a scientist who interprets the environmental impacts of the effluent release | C | C |
| The mayor says in a press release pollution can be from any company and that water tests of the river do not show elevated toxins. | I | C |
| A resident tells you he has noticed the bad smell of the river water just downstream of the company | C | C |
| You were given an undercover recording in which the CEO admits that his company has discharged untreated water into the river for years, saying everyone is doing this. | C | I |
| Water samples of the effluent of the pipeline show highly toxic discharges | C | I |
Example
This example also shows that even within the most likely scenario it is quite possible that there can be evidence that is inconsistent with your hypothesis. It is up to you to evaluate that evidence and look for explanations why the evidence is inconsistent with your hypothesis as described in the previous section.
There may be investigations in which you collect evidence that contradicts or even disproves your hypothesis. It is important to be objective and open-minded when you analyse your evidence. This is not always easy, especially if you are personally invested in your hypothesis.
Tip
It can often be helpful to write a version of your investigation for yourself that references every statement and information you include with footnotes or endnotes. This will allow you to find the relevant evidence quicker if you have to review it and also makes sure that you do not include any statements or facts for which you do not have any evidence.
The only ethical and acceptable action to take when you uncover facts and strong evidence that contradicts your working hypothesis is to accept that your initial hypothesis is wrong. Then, you either make a new/alternative one that is able to account for the new facts as objectively as possible, or accept that your current investigation will not be able to prove what you thought it would, and end your investigation. Oftentimes, this situation is more complex than just a prove-disprove end result, so it is worth spending some time evaluating your initial questions and investigation process including your methodology and tools, the evidence you have collected and any possible alternatives you might have at this point.
Example
In our example you may find out, after talking to the council, that there has been indeed an increased fish mortality and water discolouration but that their testing shows this was due lack of oxygen after a large storm washed large amount of organic matter into the river. You will need to verify this claim by looking at historical meteorological data or by consulting with experts, for instance at a university. If your assessment confirms this hypothesis you may choose to end your investigation.
Managing evidence
Information security
Information security is the practice of defending your information from being accessed or damaged by unwanted others. This is of particular importance for citizen investigators, journalists and activists who are often targeted by the subjects of their investigations. Much has been written about how to protect digital data including evidence gathered during investigations.
To learn more about these subjects start by reading:
the “Safety First!” introductory guide in this Kit
and then have a look at more detailed resources like:
Security In A Box by the Tactical Tech team,
Surveillance Self-Defense by the Electronic Frontier Foundation,
Following these guidelines will allow you to better assess your context and risks, take actions such as encrypting your hard drives and protecting your communications, in particular with confidential sources. Keep in mind that you will also need to frequently back up and keep a copy of your evidence on a separate hard drive or device and this – of course – also needs to be protected and encrypted.
Encrypting your information is not just important if you operate in politically unstable or undemocratic regions, it is something everybody should be doing to protect the privacy and safety of their sources, of themselves and their data/evidence.
Storing evidence
Digital evidence
Digital evidence is of course easy to store on your computer but it is important to keep digital copies on separate hard drives and these should be kept in a different space to your computer. Just like your internal computer hard drive, the external hard drives should be fully encrypted. If you live in an area where you face significant personal or legal risk from your investigative activities, keep some copies in a safe place outside of your own home. Keep in mind though that you may be putting other people at risk by storing evidence with them.
Physical
Storing original physical evidence is more difficult. Keep a physical copy or any original documents such as letters and, where possible, scan physical documents to create digital copies to be kept encrypted on your internal and external hard drives. Environmental samples can degrade quickly (especially water samples) and they should be processed by laboratories right away rather than kept in your home.
Evidence sheet
Complex and long investigations can result in vast amounts of evidence gathered and in some cases you might not understand the full relevance of evidence until later in your investigation. For example, an image of the polluted river with GPS coordinates could become more relevant after you receive a water testing lab report that points to a specific location. A good way to manage your evidence is by creating a summary sheet that allows you to quickly locate evidence in your collection.
Here is a sample evidence sheet that you can adapt in order to summarise information about each piece of evidence you collected.
ID: A unique ID number system you create for each piece of information. For instance, you could choose “I 0001” if it an image or “A 000”1 if this is an audio file.
Date created: The date when you have taken a video or image of the evidence, or the date you can find on a document, or the date on which you have downloaded a file or page from the internet, or the date of a social media post, etc.
File name: The name of the file if it is digital evidence.
Storage link: A link to the location of the file in the folder management on your computer.
Type: The evidence type, such as audio, image, letter, lab test, social media post, etc.
Longitude: If your evidence is linked to a geolocation you enter its longitude here.
Latitude: If your evidence is linked to a geolocation you enter its latitude here.
Location: The general location where the evidence was taken. This could be the name of a company, a village, a library or archive or any other location.
Source name: The name of the person that provided the information. This could be an interview partner, it could be yourself if you have taken a photo or the name of an office that provided you with information linked to freedom of information requests, etc.
Description: Here you include a summary of what the evidence shows.
Notes: Any other relevant details, for example any concern you might have of the accuracy or intentions of the source or other weaknesses.
Resources
Articles and Guides
Everything you wanted to know about media metadata, but were afraid to ask (Archived copy from Wayback Machine available here). By Harlo Holmes.
The Fair Use/Fair Dealing Handbook (archived copy from Wayback Machine available here). By Jonathan Band, Jonathan Gerafi.
How a Video Game Helped People Make Better Decisions (archived copy from Wayback Machine available here). By Carey K. Morewedge.
OSINT as a mindset. An article about planning your research questions, conducting thorough documentation and establishing reliability of evidence. (Archived copy from Wayback Machine available here).
Resources and Guides to Finding Expert Sources (archived copy from Wayback Machine available here). By Kira Zalan for the Global Investigative Journalism Network (GIJN).
Story-Based Inquiry – A manual for investigative Journalists (archived webpage copy from Wayback Machine available here). By Mark Lee Hunter.
Verification Handbook for Investigative Reporting (archived copy from Wayback Machine available here). By multiple authors.
Who, what, why, where? Verification of online data (archived copy from Wayback Machine available here). A Tactical Tech guide about verification methods and tools to apply when assessing online data.
Why Metadata Matters – an introduction from Surveillance Self Defense of the Electronic Frontier Foundation (archived copy available on Wayback Machine here). See the entire digital list of safety modules.
Glossary
term-cogbias
Cognitive bias - A systematic error in thinking that occurs when people are processing and interpreting information in the world around them and affects the decisions and judgements that they make (see: https://www.verywellmind.com/about-us.)
term-deskresearch
Desktop research - Research that is carried out in the home or place of work of the investigators. Examples computer and internet-based research as well as remote interviews.
term-evidence
Evidence - A wide range of information sources and investigation results that that in totality court to support or contradict an investigation hypothesis. Direct evidence establishes the main fact immediately while indirect evidence establishes facts from which the main fact can be inferred.
term-fairuse
Fair Use - A legal term that regulates the extent to which copyrighted material can re-published by third-parties.
term-fieldresearch
Field research - Research and investigative evidence gathering that takes place in locations other than in front of the computer and in the home of the investigator. This includes research in natural environments, interviews on locations, research in archives etc.
term-hypothesis
Hypothesis - A supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation (see: https://www.lexico.com/definition/hypothesis.)
term-metadata
Metadata - Information that describes properties of a file, be it image, document, sound recording, map etc. For example the contents of an image are the visible elements in it, while the date the image was taken, the location and device it was taken on, are called metadata.
term-source
Source - Publication, record or person that provides information relevant to the investigation or hypothesis.