š Policy Effectiveness: Did It Really Work?
Policy effectiveness tells us if a policy reached its intended objectives. We measure it using impact evaluation, benchmarking, and cost-benefit analysis. A policy might lower pollution, raise test scores, or reduce disease. But did the policy cause the change, or was it something else? This article explores how scientists and governments check if policies really workāwith clear examples, tables, and simple formulas.
š 1. The Counterfactual: What Would Have Happened?
To know if a policy worked, we need a counterfactualāa picture of what the world would look like without the policy. You canāt see it directly, so economists build comparison groups. Think of a new math tutoring program. If test scores rise, was it the tutoring or just a smarter class? Scientists use randomized controlled trials (RCTs)[1] to solve this, just like in medicine. Half the students get tutoring; the other half (the control group) do not. If the tutored group scores higher, the policy is effective.
ā Example: Indonesiaās INPRES school program in the 1970s built thousands of primary schools. Researchers compared regions with many new schools to regions with few. They found a clear link: more schools meant higher wages later. The policy worked.
Simple effectiveness rate:
$ \text{Effect} = (\text{Outcome}_{\text{with policy}}) - (\text{Outcome}_{\text{without policy}}) $
If the result is positive, the policy helped. If zero or negative, it didnāt workāor made things worse.
š 2. Benchmarks & Targets: Comparing to a Goal
Sometimes we compare results to a benchmarkāa fixed goal. If a city wants to reduce car crashes by 20% and installs speed cameras, effectiveness is simple: Did crashes drop at least 20%? If they dropped 15%, the policy worked partly but missed the target. Governments also use cost-effectiveness analysis: how much did we pay for each life saved, each student who passed, or each ton of pollution removed?
| Policy Example | Target (Benchmark) | Actual Result | Effective? |
|---|---|---|---|
| Speed cameras | -20% crashes | -15% crashes | ā ļø Partly |
| Flu vaccine drive | 70% vaccination rate | 68% rate | ā ļø Slightly short |
| Plastic bag tax | -50% bag use | -80% bag use | ā Yes (exceeded) |
š§Ŗ 3. Real-World Test: The Carbon Tax in British Columbia
In 2008, Canadaās province of British Columbia started a carbon tax[2]. The goal: cut fuel use and COā emissions without hurting the economy. Economists compared BC to other Canadian provinces. Result: fuel use dropped 16% more in BC than elsewhereāand the provinceās economy kept growing at the same pace. This is a famous example of a policy that achieved both environmental and economic goals.
š What if they only looked at BC alone? They might have seen less pollution and thought āsuccess!ā. But without comparing to a control group, they wouldnāt know if the whole world was simply using less fuel. Comparison proves the policy was the true cause.
ā Important Questions
A: People might give wrong answers. They might over-praise a program they like, or forget how they behaved before. Also, if everyone gets the policy, thereās no ābeforeā to compare to. Thatās why we need numbers and control groups.
A: It means the result probably isnāt a fluke. If a policy helps students gain 2 points on a test, and the statistical test says itās significant, we are confident the tutoringānot random luckācaused the gain.
A: Yes. Sometimes a policy works but costs too much. Or people dislike it (like a tax). Effectiveness is about achieving goals, not about popularity or low cost. A working policy might still be voted out.
Policy effectiveness is not just a yes/no question. It asks: Did we reach our target? Did the policy itself cause the change? Are we sure? Through counterfactuals, benchmarks, and honest comparisons, economists and governments learn what works. From building schools in Indonesia to taxing carbon in Canada, solid evidence helps us design smarter rules for a better world.
š Footnote
[1] RCT (Randomized Controlled Trial): A study where people are randomly assigned to a treatment group (gets policy) or control group (does not get policy). The gold standard for measuring effectiveness.
[2] Carbon tax: A fee on gasoline, diesel, and other fossil fuels based on how much carbon dioxide they release when burned. Designed to encourage cleaner energy.