Tests and the judge
The Train or taxi program started with the line n, a, b = map(int, input().split()), and we said we would take it apart. We will do that in this lesson, after a look at why a program reads its numbers at all instead of having them written into the code. The reason is that a judge runs your program on many different inputs.
When you submit a program to a judge such as AtCoder or Codeforces, nobody reads what it prints. Instead, the judge runs your program on a series of tests. Each test is an input with the correct output stored next to it. The judge gives your program the input and compares what it prints with the stored output.
Say you are solving a typical first problem: you are given two integers and , and you print . The problem never tells you which two integers. A program that just prints 10 would pass the test whose input is 2 5 and fail every other test, so your program has to read the numbers from its input.
The steps of one test
Here is what happens on one test of that problem, with the input 2 5:
| Step | What happens |
|---|---|
| 1 | The judge starts your program and sends it the input 2 5, as if someone had typed it. |
| 2 | The program reads the two numbers and computes . |
| 3 | The program prints 10. |
| 4 | The judge compares 10 with the correct output, 10. They match, so the test passes. |
Your submission is accepted only if it passes every test, and AtCoder shows that as AC, for accepted. If the output is different on any test, you get WA, for wrong answer. The problem statement shows you a few of the tests, called the samples. The rest are hidden, so most of the tests your program runs on are ones you never see.
Reading standard input and printing to standard output
So where does the input come from? The judge sends it to your program's standard input. That is the same place that receives what you type on the keyboard when you run a program yourself. Your answer goes out through standard output, which is where print writes. Here is a program that reads two integers from one line and prints their product. We ran it on the input 2 5:
a, b = input().split()
a = int(a)
b = int(b)
print(a * b)Let's go through it line by line. input() reads one line from standard input and gives it to you as text: '2 5', without the line break at the end. .split(), written after the text it works on, cuts that text at each space into the pieces '2' and '5', and a, b = ... gives the two pieces names, in order. The pieces are still text, though, so int(a) turns '2' into the number 2. Leave those two lines out and a * b stops with a TypeError, because Python cannot multiply two pieces of text.
Now the line from Train or taxi makes sense. a, b = map(int, input().split()) does the work of the first three lines in one go: map(int, ...) applies int to every piece; int without parentheses is the function itself, the way max is a name for a function. And if you want to print several values on one line, print(a, b) puts a space between them and shows 2 5.
This program solves AtCoder's Multiplication 1:
Two integers and are given on one line. Print their product, .
Constraints
- and are integers.
Sample 1 · Input
2 5
Output
10
Sample 2 · Input
100 100
Output
10000
The program never mentions 2 or 5, so on the second sample, 100 100, it prints 10000. You might want to make it friendlier by asking for the numbers first, with input("Enter two numbers: ").
On a judge, that prompt costs you the problem. On the input 2 5 the program would print Enter two numbers: 10, which is not 10, so the judge says WA even though the product is right. On a judge, print the answer the problem's Output section asks for and nothing else. That includes anything you printed while hunting for a bug: delete those prints before you submit.
Reading input spread over several lines
Every problem has an Input section that shows you the layout of the input: which values come first, and which line each one is on. Your program has to read the input in that layout. In this problem, the two values are on separate lines:
A user's new rating after a contest is the average of their current rating and their performance in the contest. Print the performance that makes the new rating exactly . The input gives on the first line and on the second.
Constraints
- All input values are integers.
Sample 1 · Input
2002 2017
Output
2032
Sample 2 · Input
4500 0
Output
-4500
If the performance is , then , so . That gives for the first sample and for the second, so the answer can be negative.
Read the input the way we did for a one-line problem, and the program fails:
r, g = map(int, input().split())
print(2 * g - r)The program stops on its very first line with ValueError: not enough values to unpack (expected 2, got 1). input() reads exactly one line, 4500, and splitting it gives just one piece, but there are two names to fill. The fix is to read each line with its own input(): r = int(input()), then g = int(input()), then the same print.
You don't have to submit to find this mistake: running the program on the first sample shows it. That is what the next part is about.
Samples and edge cases
Before you submit, run your program on every sample and check that it prints the sample output. You can do this on AtCoder itself: once you are signed in, each contest has a Custom Test page that runs your code with any input you paste in, such as a sample.
If you run the program on your own computer instead, it stops when it reaches input() and waits for you to type a line. Paste the sample input there and press Enter.
The samples are good at catching mistakes in how you read the input: every test is laid out like the samples, so a reading mistake that breaks a hidden test breaks the samples too. They are less good at catching a formula that is right on the samples and wrong on some other input. Counting is a problem where that happens:
Two integers and are given on one line. Print how many integers satisfy .
Constraints
- and are integers.
Sample 1 · Input
2 4
Output
3
Sample 2 · Input
10 100
Output
91
Sample 3 · Input
3 2
Output
0
From 2 to 4, the integers are 2, 3 and 4, so the answer is 3. gives only 2, because subtracting counts the steps from 2 to 4, and there is always one more integer than there are steps. So the formula is :
The third sample, 3 2, asks for the integers that are at least 3 and at most 2. There are none, and the formula gives , so all three samples pass. Before you read on, try to decide: does the formula work for every input the problem allows?
The inputs a problem allows are set by its Constraints section, which gives the range of each value. Here, and are each between 1 and 100. The hidden tests usually include the smallest and largest values in those ranges. Inputs at the boundaries, either of the constraints or of the formula (such as equal to , or just past ), are called edge cases, and they are the inputs most likely to break a program. Here are four for Counting:
| Input | Why try it | Answer | |
|---|---|---|---|
1 1 | The smallest values, and . | 1 | 1 |
1 100 | The widest range the constraints allow. | 100 | 100 |
5 2 | larger than by more than 1. | 0 | |
100 1 | The largest gap with . | 0 |
The formula fails on the last two rows. In the third sample, is exactly one more than , which is the one case where comes out to 0. Any bigger gap gives a negative count, and none of the samples has one.
Step through the edge cases
The diagram below runs the counting program on two of the samples and on 5 2. We wrote the input into the program so that you can step through it. The last input runs the fixed program: a count can't be below 0, so the answer is the larger of and 0, which is what max gives you.
When , is at least 1, so max leaves it alone. When , it is 0 or less, so max returns 0.
The fixed Counting program
The fixed program reads its input again instead of having it written in. On the edge case that broke the first version, it prints 0:
a, b = map(int, input().split())
print(max(0, b - a + 1))Judge verdicts
When the samples and your edge cases pass, you submit. On AtCoder, each contest has a Submit page, open once you are signed in, where you pick the problem and your language and paste in your program. The judge then runs every test and gives you a verdict. These are the six you will see most often, and each one points you to a different kind of mistake:
| Verdict | Short | What happened | Where to look first |
|---|---|---|---|
| Accepted | AC | Every test printed the correct output in time. | Nowhere; the problem is solved. |
| Wrong Answer | WA | The program finished, but on some test its output differs from the correct one. | The edge cases, and anything printed besides the answer. |
| Time Limit Exceeded | TLE | The program was still running when the problem's time limit, often 1 or 2 seconds a test, ran out. | How much work the program does at the largest input. |
| Runtime Error | RE | The program stopped with an error, such as dividing by 0 or reading input that is not there. | How the input is read, and operations that cannot be done. |
| Memory Limit Exceeded | MLE | The program used more memory than the problem allows. | How much data the program keeps at once. |
| Compilation Error | CE | The code could not be compiled, so no test ran. | The error message, which names the line. |
Codeforces stops at the first test your program fails and tells you its number. Test 1 there is usually the first sample, so "Wrong answer on test 1" means your program fails a test you could have run yourself. AtCoder runs every test and shows a verdict for each one.
The Rating Goal program that read two lines with one input() is a typical RE. It stopped with a ValueError while reading the input, before it ever reached the formula, and the formula was right.
Common mistakes
A prompt printed before reading, as in
input("Enter two numbers: ")orcout << "Enter two numbers: ";.- Where it breaks
- On the input
2 5, the output isEnter two numbers: 10. The judge compares it with10, finds a difference and reports WA. - The fix
- Read without a prompt, and print only what the problem's Output section asks for.
A print left in from looking for a bug, such as
print(a, b)orcout << a << " " << b << "\n";before the answer.- Where it breaks
- On
2 5the program prints2 5and then10. The judge expects only10and reports WA. - The fix
- Delete those prints before submitting.
Python only: the text pieces from
input().split()used as numbers.- Where it breaks
- With
a, b = input().split()alone,a * bon the input2 5stops withTypeError: can't multiply sequence by non-int of type 'str', anda + bprints25, the two pieces of text joined together. - The fix
- Convert each piece with
int, or read witha, b = map(int, input().split()).
Python only: one
input()call for two lines of input.- Where it breaks
- In Rating Goal,
r, g = map(int, input().split())on the input4500then0stops withValueError: not enough values to unpack: RE. - The fix
- Read the input in the layout the Input section shows: one
input()per line.
Recall Your Counting program prints the right answer on all three samples, and the judge says WA. Which inputs do you try next?
Try the inputs the samples leave out: the smallest and largest values the Constraints allow, and the boundaries of the formula. For Counting that includes larger than by more than 1, such as 5 2, where is and the answer is 0. Work out each answer by hand before running the program.
Recall A program prints `Answer: 10` on the input `2 5`, and the correct output is `10`. Which verdict does it get?
WA. The program finished, so it is not RE, but the judge compares the whole output, Answer: 10, with 10, and they differ. Print only what the Output section asks for.
Key terms
| Term | Meaning |
|---|---|
| test | An input the judge runs a program on, with its correct output. |
| sample | A test printed in the problem statement. |
| standard input | Where a program reads its input: the keyboard, or the judge's test. |
| standard output | Where a program prints; the judge compares it with the correct output. |
| Constraints | The section of a problem that gives the range of each input value. |
| edge case | An input at a boundary of the constraints or of the formula. |
| verdict | The judge's result: AC, WA, TLE, RE, MLE or CE. |
| time limit | How long a program may run on one test, often 1 or 2 seconds. |
