-
Notifications
You must be signed in to change notification settings - Fork 10
Writing tests
You can see the abstract testing algorithm in the code block below.
methods = getDynamicMethods()
for (method : methods) {
result = invoke(method)
if (not result.correct) {
break
}
}As you can see, there are some methods that are invoked and return a result. The first failed test stops the whole testing. Let's see how to implement these methods.
Locations of newly mentioned classes:
org.hyperskill.hstest.stage.StageTest
Every hs-test test should one way or another extend StageTest class (or extend another class that extends this one). You may notice that this class requires a generic type to be specified, but for dynamic tests (which this page is for) generic type is unused.
Example:
public class TicTacToeTest extends StageTest { }In Koltin you are required to pass generic argument, so you can use Any.
class TicTacToeTest : StageTest<Any>() { }Locations of newly mentioned classes:
org.hyperskill.hstest.testcase.CheckResult
CheckResult object represents the result of the test. In case of a failed test, a special message will be shown to the user that should be specified in tests. This message is called feedback.
If a user's program failed the test feedback should be shown to the user to help to resolve their problem or to find a mistake. Please, do not write feedbacks like Wrong answer or Try again - such feedbacks are useless for the user. For example, instead of Your answer is wrong you can help the user by providing clearer feedback: The number of people is wrong, should be "45", but your program printed "44". See the difference: in this case, it becomes obvious for the user that they may be somehow missed one iteration of the loop. Remember that Hyperskill is a learning platform and feedbacks of tests must help the user to spot the mistake. This task becomes quite difficult because tests know nothing about the user's program, only it's output for the provided input, so that's why tests should not compare two huge strings (outputted and correct ones) blindly to check the user's output: you cannot provide useful feedback by doing that. In tests, it's better to check the user's program output little by little to spot every mistake that could happen and provide specialized feedback for the specific error.
Examples:
Via constructor (feedback is required):
return new CheckResult(count == 42, "Count should be 42, found " + count);Via static methods:
return CheckResult.wrong("Count should be 42, found " + count);
return CheckResult.correct();Via statically imported methods:
import static org.hyperskill.hstest.testcase.CheckResult.wrong;
import static org.hyperskill.hstest.testcase.CheckResult.correct;
...
return wrong("Count should be 42, found " + count);
return correct();Locations of newly mentioned classes:
org.hyperskill.hstest.exception.outcomes.WrongAnswer
org.hyperskill.hstest.exception.outcomes.TestPassed
These exceptions are very useful if you want to finish testing being deep into several methods. It can be useful if you are under a lot of methods and don't want to continue checking everything else. For example, by parsing the user's output which is obviously should be done in a separate method because it should be called from different tests.
Throwing WrongAnswer exception will be treated like returning CheckResult.wrong(...) object. It can be used often. It is recommended to use this class everywhere instead of CheckResult.wrong(...)
Throwing TestPassed exception will be treated like returning CheckResult.correct() object. It is used rarely.
Examples:
throw new WrongAnswer("Count should be 42, found " + count);
throw new TestPassed();Locations of newly mentioned classes:
org.hyperskill.hstest.dynamic.DynamicTest
org.hyperskill.hstest.dynamic.input.DynamicTesting
The most important class for you is actually an annotation @DynamicTest - it marks the method as a test for the hs-test library. It's like a @Test annotation in JUnit. The class can be applied not only to methods but to a variable DynamicTesting or to an array DynamicTesting[]. This method should return CheckResult object.
All these variables and methods should be located inside a class that extends StageTest.
Examples:
Method declaration:
@DynamicTest
CheckResult test() {
return CheckResult.correct();
}Variable declaration:
@DynamicTest
DynamicTesting test = () -> {
return CheckResult.correct();
};Array declaration:
@DynamicTest
DynamicTesting[] tests = {
() -> CheckResult.correct(),
CheckResult::correct,
() -> CheckResult.wrong("Feedback"),
() -> {
if (count != 42) {
throw new WrongAnswer("Count should be 42, found " + count);
}
return correct();
}
};All tests are ordered by their name. Since JVM stores all variables and methods in arbitrary order this approach is used to guarantee the same order of tests. The names are compared with number-awareness: all numbers inside a method/variable name are compared as numbers, not as strings.
Examples of comparison:
test < test1
test1 < test2
test2 < test10
test_2_work < test_10_work
You can change the ordering by providing order integer argument into a @DynamicTest annotation. The default value is 0, you can set it to be lower or higher. Tests with the lower order value will be tested before ones with the higher order value. If two tests have the same order, they will be ordered by their name by the algorithm described above.
Examples:
@DynamicTest(order = -5)
@DynamicTest(order = 3)You can set a time limit for the test by providing limeLimit integer argument into a @DynamicTest annotation. The value is expected to be in milliseconds. The default value is 15000 (exactly 15 seconds). You can set it to 0 or negative values to disable the time limit completely.
Examples:
@DynamicTest(timeLimit = 10000)
@DynamicTest(timeLimit = 0)You can parameterize your tests by providing data string argument into a @DynamicTest annotation. The value is expected to be a name of a variable/method that contains/returns an array of data every item of which will be passed to the test. This variable/method should reside in the same class as the test itself.
The data argument should be used only within an annotation that is applied to a method, not to a variable or an array!
Examples:
The test test will be run 5 times, for every value of the array
int[] array = {
1, 2, 3, 4, 5
};
@DynamicTest(data = "array")
CheckResult test(int x) {
...
}More formal way of writing the same test
int[][] array = {
{1}, {2}, {3}, {4}, {5}
};
@DynamicTest(data = "array")
CheckResult test(int x) {
...
}Passing 2 arguments
Object[][] array = {
{1, "Hello"},
{2, "World"},
{3, "!"}
};
@DynamicTest(data = "array")
CheckResult test(int x, String message) {
...
}Passing an array
Object[][] array = {
{new int[] {1, 2, 3}},
{new int[] {2, 3, 4}},
{new int[] {3, 4, 5}}
};
@DynamicTest(data = "array")
CheckResult test(int[] x) {
...
}Using a method to pass data
Object method() {
...
return new Object[][] {
{1, "Hello"},
{2, "World"},
{3, "!"}
};
}
@DynamicTest(data = "method")
CheckResult test(int x, String message) {
...
}You can run the same test multiple times by providing repeat integer argument into a @DynamicTest annotation. By default, this value is equal to 1. If the data argument is present, the test will be repeated for every parametrization value. You can disable the test by setting this argument to 0.
Examples:
Repeat 5 times:
@DynamicTest(repeat = 5)Disable the test
@DynamicTest(repeat = 0)Generate 15 tests, repeat 5 times for every data value
Object[][] array = {
{1, "Hello"},
{2, "World"},
{3, "!"}
};
@DynamicTest(repeat = 5, data = "array")
CheckResult test(int x, String message) {
...
}You can also create external files for testing. Before starting the test case, these files are created on the file system so that the student's program can read them. After the test finishes, these will be deleted from the file system.
The value is expected to be a name of a variable/method that contains/returns a Map<String, String> which contains filenames as keys and contents of these files as values. This variable/method should reside in the same class as the test itself.
An example of how to set up external files for test cases can be seen below:
Map<String, String> filesMap = Map.of(
"file1.txt", "content of file1.txt",
"file2.txt", "content of file2.txt"
);
@DynamicTest(files = "filesMap")
CheckResult test() {
...
}You can also use a method:
public Map<String, String> getFiles() {
return Map.of(
"file1.txt", "content of file1.txt",
"file2.txt", "content of file2.txt"
);
}
@DynamicTest(files = "getFiles")
CheckResult test() {
...
}Locations of newly mentioned classes:
org.hyperskill.hstest.testing.TestedProgram
Class TestedProgram is used to store the user's program and execute it. You should create instances of this class only inside a test. You don't need to provide any link to the user program's main method or class: it will be done automatically. The hs-test library will search for the class with the main method among all the classes the user has written.
In case there is 0 or more than 1 main method the testing will fail with the useful feedback to the user suggesting to check main methods of their programs.
@DynamicTest
CheckResult test() {
TestedProgram pr = new TestedProgram();
...
}Sometimes the project requires to have more than 1 main method (for example, chat with server and client both having the main method). For this, you need to pass a string to the constructor representing a package where to search for the main method. The search is done recursively, considering inner packages also.
@DynamicTest
CheckResult test() {
TestedProgram server = new TestedProgram("chat.server");
TestedProgram client = new TestedProgram("chat.client");
...
}Creating TestedProgram instance the user's program won't be started. Moreover, despite having already analyzed the user's program for the existence of the main method, all the classes the user has written won't be initialized: all classes' static members won't be instantiated and no static blocks in these classes will be executed.
Every instance of TestedProgram uses a separate own instance of custom ClassLoader to load the user's classes to Java's memory. It means, that every test will trigger initialization of static members and execution of classes' static blocks. This ensures that every test will be run like it is being run for the first time, and the result of the test can be repeated by manually running the user's program.
Note that if some test runs two programs simultaneously (client and server) they cannot interact in any way because they were initialized by different ClassLoaders: server will see empty fields in client's classes and client will see empty fields in server's classes. Much like in real life, when these two programs will be run on different computers.
To start tested program, you need to invoke either .start(...) or .startInBackground(...) methods. The first one is used almost all the time and the second one is used rarely. You can pass command-line arguments to the tested program as the parameters of these methods. The execution of the test will pause at this point and user's program will be launched. It will be run until it requests input. After that, it will be paused and the execution of the test resumes. The .start(...) method will return the output user's program printed before requesting an input.
Example:
class Main {
public static void main(String[] args) {
System.out.println(args[0] + " " + args[1]);
String line = new Scanner(System.in).nextLine();
...
}
}
...
@DynamicTest
CheckResult test() {
TestedProgram pr = new TestedProgram();
String output = pr.start("Hello", "World!");
...
}Let's discuss the code above step by step.
- The
new TestedProgram()part will search the user's program for the main method and that's it. - The
pr.start("Hello", "World!")will start the user's program with two command-line arguments:"Hello"and"World!". Then the execution of the test pauses. - The user's program starts to execute. The
System.out.println(args[0] + " " + args[1])will print to theSystem.outthe text"Hello World!\n". This text will be saved by thehs-test. - The
new Scanner(System.in).nextLine()will request an input that is not determined at the moment, so the execution of the user's program pauses and the execution of the test resumes. - The
pr.start("Hello", "World!")will return output that was collected while the program was running. So theoutputvariable will be equal to"Hello World!\n".
Example 2:
class Main {
public static void main(String[] args) {
System.out.println(args[0] + " " + args[1]);
String line = new Scanner(System.in).nextLine();
...
}
}
...
@DynamicTest
CheckResult test() {
TestedProgram pr = new TestedProgram();
pr.startInBackground("Hello", "World!");
...
}The only difference with the previous example is that we use .startInBackground(...) method instead of .start(...). The execution differences are:
- It returns to the test execution immediately and the tested program will be executed in parallel (note that in the previous example everything is executed consecutively).
- The method returns nothig because it is declared as
void.
We consider ways to extract output from the program that runs in the background later.
After returning from the .start(...) method you need to generate input for the user's program and pass it to the program using the .execute(...) method. As the .start(...) method, the .execute(...) method will pause test execution and continue the user's program until it runs out of input. After that, the user's program will be paused and the execution of the test resumes. The .execute(...) method will return output that the user's program has printed in this period.
Additionally, .start(...) and .execute(...) methods will return output not only in case the user's program requests additional input, but also in case of finishing the user's program. In case of additional .execute(...) call on this tested program the execution will immediately fail and the user will see feedback Program is finished too early. Additional input request was expected.
Use .isFinished() method to check if the program is finished or not. You can also call .stop() to terminate the user's program but it's not required to do so: every TestedProgram instance that was started will be stopped anyway after the end of the test.
Example:
class Main {
public static void main(String[] args) {
System.out.println(args[0] + " " + args[1]);
String line = new Scanner(System.in).nextLine();
if (line.equals("42")) {
System.out.println("Life");
} else {
System.out.println("Not life");
}
line = new Scanner(System.in).nextLine();
...
}
}
...
@DynamicTest
CheckResult test() {
TestedProgram pr = new TestedProgram();
String output = pr.start("Hello", "World!");
if (!output.contains("Hello") || !output.contains("World!")) {
throw new WrongAnswer("Your output should contain " +
"\"Hello\" and \"World!\", but it isn't");
}
output = pr.execute("42").toLowerCase().trim();
if (!output.equals("life")) {
throw new WrongAnswer("Your output should be equal to " +
"\"Life\" after inputting \"42\"");
}
output = pr.execute("Line1\nLine2\nline3\n");
...
}Let's discuss the code above step by step following step #5 from the example in the previous section.
- Between invocations of
.start(...)and.execute(...)or between two invocations of.execute(...)methods you need to check the user's output. It is very convenient to check these partial outputs compared to checking the whole output at once. This way it is possible to check different aspects of the user's output and generate feedbacks that are useful for the user. Thus, in the test we check that the user's output contains"Hello"and"World!"(for example, it is required in the stage description). - If the check fails, the
throw new WrongAnswer(...)will throw an exception and stops the test. After that, thehs-testlibrary will stop the user's program automatically. - The
pr.execute("42")will pass the string"42"as the input to the user's program and waits till the user's program consumes all of it and requests additional input. - Then the user's program resumes and outputs
"Life". After that the program requests additional input which is undetermined. Thus, the user's program pauses and the execution of the test resumes. - The
pr.execute("42")returns"Life\n"and processed to be"life"using.toLowerCase().trim()methods. Actually, it is a good idea to process lowercased and trimmed output strings because more formally correct solutions will pass these tests. Tests should not check the output too strictly and should allow some degree of freedom especially in parts where it's not so important. - The
pr.execute("Line1\nLine2\nline3\n")passes 3 lines to the user's program. And returns only when the user's program consumes all 3 lines (or exits). - The further checking continues the same way.
You can create a background program by launching it using the method .startInBackground(...). But actually, you can move the tested program to the background by calling the method .goBackground(). Upon calling this method:
- It immediately returns to the test method.
- The user's program continues its execution in parallel. But it will still wait for the input since it was waiting before this call. The difference is that after the following
.execute(...)call it won't wait for the user's program and returns immediately.
The other method does the opposite: it's called .stopBackground(). Upon calling this method:
- It immediately returns if the program not in the background.
- Otherwise it waits for the first input request from the user's program and then returns like it's not in the background anymore. And it really is not.
Use .isInBackground() to check if the program is executing in the background or not.
If the tested program executes in the background, .startInBackground(...) call and any .execute(...) call will return immediately and return an empty string. The only way to retrieve the user's program output is to call .getOutput() method which will return the output that was collected since the previous .getOutput() call.
You can use .getOutput() call if the program not in the background also. It will return output that was printed since the last .execute(...) call. It might not be empty because the user's program can print some text in a separate thread. It will always be empty in case the user's program is single-threaded.
If the user's program is running in the background and not waiting for the input, any .execute(...) call will fail the testing and hs-test library will blame the test author for such an outcome. To be sure that the user's program waits for the output use the .isWaitingInput() method. Alternatively, you can use .stopBackground() and .goBackground() calls consecutively.
Sometimes, you need to check output once after several .execute(...) calls. The only way to achieve that is to collect output after every .execute(...) call and it can be inconvenient and error-prone. It could be done in the following way:
output = pr.execute(...)
outout += pr.execute(...)
outout += pr.execute(...)
...
if (?) {
output += pr.getOutout()
}
check(output)Also, sometimes the tested program uses background threads to print some information and it just so happens that input request was performed just before such output, so .exexute(...) call won't return such line but tests expect it to do so. Sometimes this output will be returned inside the .execute(...) statement, sometimes you should call .getOutput() to get output that was printed after the last .execute(...) call.
The hs-test library guarantees that the sum of all .start(...), .execute(...), .getOutput() calls will result in a single output with no duplicate lines or characters, so one outputted character must appear only once among these calls. But as was demonstrated above, sometimes it is really hard to determine which call contains a result you need to check.
Upon calling .setReturnOutputAfterExecution(false) method:
- Any
.start(...)and.execute(...)calls will return an empty string (but they will still wait for the input request if they are not in the background mode). - The only way to retrieve the user's program output is to call
.getOutput(). This call will return all the output since the last.getOutput()call.
This way the hs-test library ensures that the sum of all .start(...), .execute(...), .getOutput() calls will result in a single output but now you can be sure that the output you need to check is inside a certain .getOutput() call.
You can see the updated code below:
pr.setReturnOutputAfterExecution(false)
...
pr.execute(...)
pr.execute(...)
pr.execute(...)
...
check(pr.getOutput())The user's program:
1 class Main {
2 public static void main(String[] args) {
3 // some code that print something
4
5 Scanner scanner = new Scanner(System.in);
6 int num = scanner.nextInt();
7
8 // some code that process input and print something
9
10 String line = scanner.nextLine();
11
12 // some code that process input and print something
13 // ...
14 }
15 }The tests (see comments for explanations):
import org.hyperskill.hstest.dynamic.DynamicTest;
import org.hyperskill.hstest.exception.outcomes.TestPassed;
import org.hyperskill.hstest.exception.outcomes.WrongAnswer;
import org.hyperskill.hstest.stage.StageTest;
import org.hyperskill.hstest.testcase.CheckResult;
import org.hyperskill.hstest.testing.TestedProgram;
public class TestUserProgram extends StageTest<String> {
@DynamicTest
CheckResult test() {
TestedProgram main = new TestedProgram();
// You can pass command-line args here as parameters
// This output is from the start of the program
// execution till the first input (lines 3, 4, 5)
String output = main.start().toLowerCase();
if (!output.contains("hello")) {
// You can return CheckResult object and
// the execution will stop immediately
return CheckResult.wrong(
"Your program should greet the user " +
"with the word \"Hello\"");
}
if (!output.contains("input")) {
// You can throw WrongAnswer error here also,
// like in the 'check' method
throw new WrongAnswer(
"Your program should ask the user " +
"to print something (output the word \"input\")");
}
// If you want to continue to execute the user's program
// you should invoke 'execute' method and pass input as a string.
// You can pass multiple lines, in this case
// execution of this method continues only when
// all the lines will be consumed by the user's program
// and additional input will be requested.
String output2 = main.execute("42");
// This output is from the first input input
// till the second input (lines 6, 7, 8, 9)
output2 = output2.toLowerCase();
if (output2.contains("42")) {
// You may also want to stop execution but indicate
// that the user's program is correct on this test
return CheckResult.correct();
}
if (output2.contains("4+2")) {
// You can throw TestPassed error here also,
// like in the 'check' method to indicate
// that the user's program is correct on this test
throw new TestPassed();
}
String output3 = main.execute("line1\nline2");
// Now you can test all the output here and not in 'check' method
// usually, checking all the output parts is enough, but to fully
// simulate previous example you can write the following code:
String reply = output + output2 + output3;
if (reply.trim().split("\n").length != 4) {
return CheckResult.wrong("Your program should print exactly 4 lines");
}
if (false) {
// You can force to additionally invoke 'check' method
// and test all the output there by returning null in this method
// but usually, it's not needed at all since
// you can fully check all the output parts in this method
return null;
}
// You can check whether the main method is finished or not
// Don't worry if you are feel you need to call this method
// It's automatically checked in every "execute" call
// and also after finishing this dynamic method
if (!main.isFinished()) {
return CheckResult.wrong(
"Your program should not request more, than 3 lines.");
}
return CheckResult.correct();
}
// Other examples:
@DynamicTest
CheckResult test2() {
// you can use multiple methods marked with this
// annotation, but the execution order is not guaranteed
// to be from top to bottom, keep it in mind.
// Dynamic testing methods are sorted alphabetically by their method name.
// with a slight improvement that numbers compared as numbers
// so "test2" will execute earlier than "test10"
return CheckResult.correct();
}
@DynamicTest
CheckResult test3() {
TestedProgram main = new TestedProgram();
main.start();
Random r = new Random();
// a big advantage of this approach that you can use
// loops with an undetermined number of iterations.
int tries = 0;
while (true) {
// test tries to guess a number from 0 to 9
String output = main.execute("" + tries);
if (output.contains("you guessed")) {
break;
}
tries++;
// but keep in mind that the user can mess up
// and stuck in a guessing loop infinitely
if (tries == 10) {
throw new WrongAnswer(
"Test tried all the possible numbers but " +
"didn't guess the number");
}
}
// another advantage is that you can use different number
// of executions in different branches (this is impossible
// to simulate using dynamic input functions)
if (true) {
main.execute("");
main.execute("");
} else {
main.execute("");
}
return CheckResult.correct();
}
@DynamicTest
CheckResult test4() {
// You can also test several programs simultaneously
// for example, it can be server and couple of clients
TestedProgram server = new TestedProgram("chat.server");
TestedProgram client1 = new TestedProgram("chat.client");
TestedProgram client2 = new TestedProgram("chat.client");
// Since server should actually listen for clients
// and not request for an input you can launch it
// in the background
server.startInBackground();
// Clients will start in a blocking way meaning
// only one of them will execute its code at a time
// one after another and inside this method in between
client1.start();
client2.start();
// Let's say clients can login to the server using login
String out = client1.execute("login admin");
if (!out.trim().equals("Enter password:")) {
// All the clients and server will be automatically
// shut down after exiting this method
return CheckResult.wrong(
"Can't see if client requested the password");
}
client1.execute("qwerty");
if (!out.toLowerCase().contains("wrong password")) {
return CheckResult.wrong(
"\"qwerty\" is wrong password for the admin");
}
client1.execute("f7ryaeuhd87");
client2.execute("login super_nick");
client2.execute("qwerty");
// ...
return CheckResult.correct();
}
}Dynamic input is an older approach to write such tests with dynamically generated input. Note, that approach described above is called Dynamic method, you should use it to write tests.
Dynamic input is now deprecated. So, don't write the tests using the methods described below, use this information just to understand the tests that were written in such a way.
Dynamic input is based on a set of dynamic functions that are called in the process of the user's program execution. When the user's program asks for input and there is no input left, the next dynamic function is called to which the output the program has printed from the previous dynamic function call is passed.
To add a dynamic function to the set in Java you need to call .addInput(..) which takes a function that takes a String (it's user's program output so far printed).
A dynamic function may:
- Return a
String, which is considered to be the next input to be passed to the user's program. - Return a
CheckResultobject. In this case, the execution of the user's program immediately stops, and the result of this test is determined by this object. - Throw a
TestPassedor aWrongAnswerexception. In this case, the execution of the user's program also immediately stops, and the result of this test is determined by this exception.
By default, dynamic functions can be executed only once, but if you want them to be triggered multiple times, you can pass a number along with the function. In case you pass a negative number, the function will be executed infinitely. Instead of a function, you can also pass a String, it's a shortcut to a function that always returns this string as the next input.
Examples of defining dynamic functions
Java
new TestCase<String>()
.addInput("input1") // Just returns "input1" as the next input once
.addInput(12, "input2") // Just returns "input2" as the next input 12 times
.addInput(output -> { // Returns "input3" in case user's program printed "hello" before requesting this input
if (!output.toLowerCase().trim().contains("hello")) {
return CheckResult.wrong("Greet the user!");
}
return "input3";
})
.addInput(5, out -> out + "input4") // Returns 'out + "input4"' 5 times
.addInput(String::toLowerCase) // Returns out.toLowerCase() once
.addInput(-1, this::testFunc); // Returns this.testFunc(out) infinitely (the following dynamic functions will never be executed)Let's see a more complex example:
User's program in Java
1 class Main {
2 public static void main(String[] args) {
3 // some code that print something
4
5 Scanner scanner = new Scanner(System.in);
6 int num = scanner.nextInt();
7
8 // some code that process input and print something
9
10 String line = scanner.nextLine();
11
12 // some code that process input and print something
13 // ...
14 }
15 }Test in Java (see comments for explanations):
import org.hyperskill.hstest.exception.outcomes.TestPassed;
import org.hyperskill.hstest.exception.outcomes.WrongAnswer;
import org.hyperskill.hstest.stage.StageTest;
import org.hyperskill.hstest.testcase.CheckResult;
import org.hyperskill.hstest.testcase.TestCase;
import java.util.Arrays;
import java.util.List;
public class TestUserProgram extends StageTest<String> {
@Override
public List<TestCase<String>> generate() {
return Arrays.asList(
new TestCase<String>()
.addInput(output -> {
// This output is from the start of the program
// execution till the first input (lines 3, 4, 5)
output = output.toLowerCase();
if (!output.contains("hello")) {
// You can return CheckResult object and
// the execution will stop immediately
return CheckResult.wrong(
"Your program should greet the user " +
"with the word \"Hello\"");
}
if (!output.contains("input")) {
// You can throw WrongAnswer error here also,
// like in the 'check' method
throw new WrongAnswer(
"Your program should ask the user " +
"to print something (output the word \"input\")");
}
// If you want to continue to execute the user's program
// you should return an input as a string.
// You can return multiple lines, in this case
// the next dynamic input function will be invoked only when
// all the lines will be consumed by the user's program
// and additional input will be requested.
return "42";
})
.addInput(output -> {
// This output is from the first input input
// till the second input (lines 6, 7, 8, 9)
output = output.toLowerCase();
if (output.contains("42")) {
// You may also want to stop execution but indicate
// that the user's program is correct on this test
return CheckResult.correct();
}
if (output.contains("4+2")) {
// You can throw TestPassed error here also,
// like in the 'check' method to indicate
// that the user's program is correct on this test
throw new TestPassed();
}
return "line1\nline2";
})
);
}
@Override
public CheckResult check(String reply, String attach) {
// If no CheckResult object was returned or no error was thrown
// from every invoked dynamic input function then 'check'
// method is still invoked. "reply" variable contains all the output
// the user's program printed.
if (reply.trim().split("\n").length != 4) {
return CheckResult.wrong("Your program should print exactly 4 lines");
}
return CheckResult.correct();
}
}- Home
- About
- Initial setup
- Writing tests
- Guidelines for writing tests
- Outcomes of testing
- Generating and checking
- Presentation error
- Checking JSON
- Testing solutions written in different languages
- Creating Hyperskill problems based on hs-test
- Testing Java Swing applications
- Testing Java Spring applications
- Testing Ktor applications