When writing unit tests, having assertions is a fundamental requirement. They are, ultimately, the "test" part of a unit test. But many developers are unfamiliar with the assertion libraries that come with the popular unit testing frameworks, so don't get the full range of their benefits. Too many times in my career, I've seen developer exclusively utilize Assert.True(...)
with some conditional check inside it. And while this will fail the test when the conditional is no longer met, all you get from the failure is a message that says:
Expected: True But was: FalseWhich means to even figure out what's wrong, you have to, at minimum, look at the unit test code to see what it was even testing.
My hope is that, after reading through this post (and bookmarking it for frequent reference), you'll be able to write unit tests that easily provide you with enough detail in the failure messages that you can make some educated guesses about why it's failing without ever looking at the code.
For example, here's some NUnit failure messages -- can you guess what the test was attempting to validate?
Expected: 100 +/- 8 Percent But was: 91
Expected: equivalent to < "bar", "baz" > But was: < "foo", "bar", "baz", "bin", <string.Empty> > Extra (3): < "foo" , "bin" , <string.Empty> >
In this post, I cover the assertion options and syntax for each of the three most popular .NET unit testing frameworks: NUnit, xUnit and MSTest. Plus, I throw in a few personal opinions about the frameworks along the way.
The Framework Options
Here are the frameworks I'm going to focus on in this post:
- NUnit
Note: NUnit has two variants of it's syntax, which I will call "NUnit Constraint" (or just "NUnit") and "NUnit Classic" styles. - XUnit.net
- MSTest (Visual Studio)
- Shouldly
- Fluent Assertions
Initial Thoughts
Personally, I greatly prefer the NUnit Constraint-style asserts and will push to use NUnit on any project I can because of this. NUnit's Constraint-style assert syntax is easy to read, has the most useful out-of-the-box failure messages (particularly for dealing with collections), and has more features built-in than the other built-in framework options.
xUnit.net has become the preferred testing framework for the Microsoft .NET codebase, and the framework itself has some nice features, but it still lags a bit behind NUnit in my opinion.
MSTest is easily my least favorite, as it has the worst documentation (though it's improving), the least features, and the syntax is a bit clunky. But it has the huge benefit of being built-in to Visual Studio, so takes the least effort to get up-and-running, though NuGet packages make the other two extremely simple as well.
Shouldly's main difference is that it adds extension methods off of your normal objects. So instead of calling Assert...
, you would write myVar.Should...
. Additionally, Shouldly uses reflection to inspect the code around your call to provide more useful information, like the variable name and LINQ statements called. However, it seems quite a bit behind where Fluent Assertions is from a maturity standpoint, and I would suggest going with Fluent Assertions over Shouldly.
Fluent Assertions comes in second place. Like Shouldly, it uses fluent-style extension methods instead of Assert.That...
calls. While I haven't used it (yet) on a production codebase, just "playing" with it for this blogpost and related GitHub repo has shown me that it's an easy to use and full-featured framework. And it's documentation is very well done. I highly suggest you give it a review if you're getting started with a project.
All of these frameworks support both the .NET Full Framework and .NET Core/netstandard, so you shouldn't have any issues using them across all your projects.
How Asserts Work
The implementation for asserts are pretty straightforward. Unit tests will fail if an unexpected and uncaught Exception occurs. The failure message reported by the testing framework is the Exception.Message
value. (Plus, most framework runners will also show you the full stacktrace of the Exception).
So Asserts are just shortcuts for throwing an Exception when a comparison isn't true. In most cases, the Assert methods will take in an "expected" value (ie: what you want the result to be if your code is working correctly), an "actual" value (ie: the value your code actually generated), and an optional "message". Given those inputs, the Assert method will compare the actual value to the expected value and if the comparison fails, it will generated an Exception with a Message field that contains some hopefully useful information about the actual and expected values and the comparison attempted, as well as the additional "message" if you provided one.
So if you take the case of Assert.That(actual, Is.GreaterThan(5))
, you could just as well write this code to get the same result:
if (actual < 5) throw new Exception("Expected a value greater than 5, but got " + actual)
Syntax Examples
When it comes to syntax, the NUnit Constraint-style syntax is different than the other test framework built-in options -- something I think makes it much more readable and usable. xUnit.net, MSTest and the NUnit Classic-style assertions all follow the pattern of Assert.Something(expectedValue, actualValue)
. The NUnit Constraint-style syntax attempts to read more like an English-language sentence, like Assert.That(actualValue, Is.SomethingTo(expectedValue))
.
Both Shouldly and Fluent Assertions take a different approach and use a fluent-style syntax that starts with the actual value from your test execution and compares it to the expected values in an English-like way. (ex "myValue should be greater than 0"). They also use Reflection to try and provide additional context in the error messages upon failure.
A word about custom messages
All of the assertion options except xUnit.net allow you to provide a custom message to show in addition to the assertion's own output upon failure. This message optional but is the most effective way of providing useful output when your tests fail, since you can add whatever data you deem important at the time you're writing the test.
But, we're all generally pretty lazy when it comes to writing unit tests, and in my experience we'll only include a custom message on rare occasions. The rest of the time, we rely on the default output from the assertions themselves.
In those cases where you do want to provide your own messages, the assertion methods take a final string parameter that you can use to provide that message, like this: Assert.GreaterOrEqual(0, price, "Price should never be less than 0");
. This way your output can explain the failure with output like this:
Price should never be less than 0 Expected: greater than or equal to 0 But was: -2.52
Equality Checks
This first set of equality checks are effectively checking object.Equals()
to determine if the two values are equal.
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [Test] public void EqualityChecks() { bool valueToTest_bool = true; string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; Assert.That(valueToTest_bool, Is.EqualTo(true)); Assert.That(valueToTest_string, Is.EqualTo("some result")); Assert.That(valueToTest_datetime, Is.EqualTo(new DateTime(2019, 01, 01))); Assert.That(valueToTest_obj, Is.EqualTo(expectedValue_obj_equal)); Assert.That(valueToTest_bool, Is.Not.EqualTo(false)); Assert.That(valueToTest_string, Is.Not.EqualTo("some other result")); Assert.That(valueToTest_datetime, Is.Not.EqualTo(new DateTime(2019, 12, 01))); Assert.That(valueToTest_obj, Is.Not.EqualTo(expectedValue_obj_notequal)); }
Expected: <{ Foo=zoom, Baz=False }> But was: <{ Foo=bar, Baz=True }> Expected: not equal to 2019-01-01 00:00:00 But was: 2019-01-01 00:00:00 Expected string length 17 but was 11. Strings differ at index 5. Expected: "some other result" But was: "some result" ----------------^
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [Test] public void EqualityChecks() { bool valueToTest_bool = true; string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; // (important: expected value comes first!) Assert.AreEqual(true, valueToTest_bool); Assert.AreEqual("some result", valueToTest_string); Assert.AreEqual(new DateTime(2019, 01, 01), valueToTest_datetime); Assert.AreEqual(expectedValue_obj_equal, valueToTest_obj); Assert.AreNotEqual(false, valueToTest_bool); Assert.AreNotEqual("some other result", valueToTest_string); Assert.AreNotEqual(new DateTime(2019, 12, 01), valueToTest_datetime); Assert.AreNotEqual(expectedValue_obj_notequal, valueToTest_obj); }
Expected: <{ Foo=zoom, Baz=False }> But was: <{ Foo=bar, Baz=True }> Expected: not equal to 2019-01-01 00:00:00 But was: 2019-01-01 00:00:00 Expected string length 17 but was 11. Strings differ at index 5. Expected: "some other result" But was: "some result" ----------------^
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [Fact] public void EqualityChecks() { string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; // (important: expected value comes first!) Assert.Equal("some result", valueToTest_string); Assert.Equal(new DateTime(2019, 01, 01), valueToTest_datetime); Assert.Equal(expectedValue_obj_equal, valueToTest_obj); Assert.NotEqual("some other result", valueToTest_string); Assert.NotEqual(new DateTime(2019, 12, 01), valueToTest_datetime); Assert.NotEqual(expectedValue_obj_notequal, valueToTest_obj); }
Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: { Foo = zoom, Baz = False } Actual: { Foo = bar, Baz = True } Xunit.Sdk.NotEqualException: Assert.NotEqual() Failure Expected: Not 2019-01-01T00:00:00.0000000 Actual: 2019-01-01T00:00:00.0000000 Xunit.Sdk.EqualException: Assert.Equal() Failure ↓ (pos 5) Expected: some other result Actual: some result ↑ (pos 5)
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [TestMethod] public void EqualityChecks() { bool valueToTest_bool = true; string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; // (important: expected value comes first!) Assert.AreEqual(true, valueToTest_bool); Assert.AreEqual("some result", valueToTest_string); Assert.AreEqual(new DateTime(2019, 01, 01), valueToTest_datetime); Assert.AreEqual(expectedValue_obj_equal, valueToTest_obj); Assert.AreNotEqual(false, valueToTest_bool); Assert.AreNotEqual("some other result", valueToTest_string); Assert.AreNotEqual(new DateTime(2019, 12, 01), valueToTest_datetime); Assert.AreNotEqual(expectedValue_obj_notequal, valueToTest_obj); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<{ Foo=zoom, Baz=False }>. Actual:<{ Foo=bar, Baz=True }>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreNotEqual failed. Expected any value except:<1/1/2019 12:00:00 AM>. Actual:<1/1/2019 12:00:00 AM>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<some other result>. Actual:<some result>.
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [Test] public void EqualityChecks() { bool valueToTest_bool = true; string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; valueToTest_bool.Should().Be(true); valueToTest_string.Should().Be("some result"); valueToTest_datetime.Should().Be(new DateTime(2019, 01, 01)); valueToTest_obj.Should().Be(expectedValue_obj_equal); valueToTest_string.Should().NotBe("some other result"); valueToTest_datetime.Should().NotBe(new DateTime(2019, 12, 01)); valueToTest_obj.Should().NotBe(expectedValue_obj_notequal); }
Expected valueToTest_obj to be { Foo = zoom, Baz = False }, but found { Foo = bar, Baz = True }. Expected valueToTest_datetime not to be <2019-01-01>, but it is. Expected valueToTest_string to be "some other result" with a length of 17, but "some result" has a length of 11, differs near "res" (index 5).
/// <summary> /// Assertions that test for equality using object.Equals() to compare the actual value /// to the expected value /// </summary> [Test] public void EqualityChecks() { bool valueToTest_bool = true; string valueToTest_string = "some result"; DateTime valueToTest_datetime = new DateTime(2019, 01, 01); var valueToTest_obj = new { Foo = "bar", Baz = true }; var expectedValue_obj_equal = new { Foo = "bar", Baz = true }; var expectedValue_obj_notequal = new { Foo = "zoom", Baz = false }; ; valueToTest_bool.ShouldBe(true); valueToTest_string.ShouldBe("some result"); valueToTest_datetime.ShouldBe(new DateTime(2019, 01, 01)); valueToTest_obj.ShouldBe(expectedValue_obj_equal); valueToTest_bool.ShouldNotBe(false); valueToTest_string.ShouldNotBe("some other result"); valueToTest_datetime.ShouldNotBe(new DateTime(2019, 12, 01)); valueToTest_obj.ShouldNotBe(expectedValue_obj_notequal); }
Shouldly.ShouldAssertException : valueToTest_obj should be { Foo = zoom, Baz = False } but was { Foo = bar, Baz = True } Shouldly.ShouldAssertException : valueToTest_datetime should be 2019-02-01T00:00:00.0000000 but was 2019-01-01T00:00:00.0000000 Shouldly.ShouldAssertException : valueToTest_string should be "some other result" but was "some result" difference Difference | | | | | | | | | | | | | | \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ Index | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Expected Value | s o m e \s o t h e r \s r e s u l t Actual Value | s o m e \s r e s u l t Expected Code | 115 111 109 101 32 111 116 104 101 114 32 114 101 115 117 108 116 Actual Code | 115 111 109 101 32 114 101 115 117 108 116
This second set of equality checks are utilizing object.ReferenceEquals()
to determine if the two objects are actually referring to the same exact object in memory
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [Test] public void SameObjectChecks() { var valueToTest = new {Foo = "bar", Baz = true}; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; Assert.That(valueToTest, Is.SameAs(expectedValue_same)); Assert.That(valueToTest, Is.Not.SameAs(expectedValue_notsame)); }
Expected: same as <{ Foo=bar, Baz=True }> But was: <{ Foo=bar, Baz=True }> Expected: not same as <{ Foo=bar, Baz=True }> But was: <{ Foo=bar, Baz=True }>
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [Test] public void SameObjectChecks() { var valueToTest = new {Foo = "bar", Baz = true}; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; // (important: expected value comes first!) Assert.AreSame(expectedValue_same, valueToTest); Assert.AreNotSame(expectedValue_notsame, valueToTest); }
Expected: same as <{ Foo=bar, Baz=True }> But was: <{ Foo=bar, Baz=True }> Expected: not same as <{ Foo=bar, Baz=True }> But was: <{ Foo=bar, Baz=True }>
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [Fact] public void SameObjectChecks() { var valueToTest = new { Foo = "bar", Baz = true }; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; // (important: expected value comes first!) Assert.Same(expectedValue_same, valueToTest); Assert.NotSame(expectedValue_notsame, valueToTest); }
Xunit.Sdk.SameException: Assert.Same() Failure Expected: { Foo = bar, Baz = True } Actual: { Foo = bar, Baz = True } Xunit.Sdk.NotSameException: Assert.NotSame() Failure
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [TestMethod] public void SameObjectChecks() { var valueToTest = new { Foo = "bar", Baz = true }; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; // (important: expected value comes first!) Assert.AreSame(expectedValue_same, valueToTest); Assert.AreNotSame(expectedValue_notsame, valueToTest); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreSame failed. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreNotSame failed.
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [Test] public void SameObjectChecks() { var valueToTest = new {Foo = "bar", Baz = true}; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; valueToTest.Should().BeSameAs(expectedValue_same); valueToTest.Should().NotBeSameAs(expectedValue_notsame); }
Expected valueToTest to refer to { Foo = bar, Baz = True }, but found { Foo = bar, Baz = True }. Did not expect valueToTest to refer to { Foo = bar, Baz = True }.
/// <summary> /// Assertions that test for equality using object.ReferenceEquals() to determine if both values point to /// the exact same object. /// </summary> [Test] public void SameObjectChecks() { var valueToTest = new {Foo = "bar", Baz = true}; var expectedValue_same = valueToTest; var expectedValue_notsame = new { Foo = "bar", Baz = true }; ; Assert.That(valueToTest, Is.SameAs(expectedValue_same)); Assert.That(valueToTest, Is.Not.SameAs(expectedValue_notsame)); }
Shouldly.ShouldAssertException : valueToTest should be same as { Foo = bar, Baz = True } but was { Foo = bar, Baz = True } Shouldly.ShouldAssertException : valueToTest should not be same as { Foo = bar, Baz = True } but was { Foo = bar, Baz = True }
Comparision Checks
This first set will check your value against null
/// <summary> /// Assertions that test for null values /// </summary> [Test] public void NullChecks() { var valueToTest = new { Foo = (object) null, Baz = new object() }; Assert.That(valueToTest.Foo, Is.Null); Assert.That(valueToTest.Baz, Is.Not.Null); }
Expected: not null But was: null Expected: null But was: <System.Object>
/// <summary> /// Assertions that test for null values /// </summary> [Test] public void NullChecks() { var valueToTest = new { Foo = (object) null, Baz = new object() }; Assert.Null(valueToTest.Foo); Assert.IsNull(valueToTest.Foo); Assert.NotNull(valueToTest.Baz); Assert.IsNotNull(valueToTest.Baz); }
Expected: not null But was: null Expected: null But was: <System.Object>
/// <summary> /// Assertions that test for null values /// </summary> [Fact] public void NullChecks() { var valueToTest = new { Foo = (object)null, Baz = new object() }; Assert.Null(valueToTest.Foo); Assert.NotNull(valueToTest.Baz); }
Xunit.Sdk.NotNullException: Assert.NotNull() Failure Xunit.Sdk.NullException: Assert.Null() Failure Expected: (null) Actual: Object { }
/// <summary> /// Assertions that test for null values /// </summary> [TestMethod] public void NullChecks() { var valueToTest = new {Foo = (object) null, Baz = new object()}; Assert.IsNull(valueToTest.Foo); Assert.IsNotNull(valueToTest.Baz); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsNotNull failed. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsNull failed.
/// <summary> /// Assertions that test for null values /// </summary> [Test] public void NullChecks() { var valueToTest = new { Foo = (object) null, Baz = new object() }; valueToTest.Foo.Should().BeNull(); valueToTest.Baz.Should().NotBeNull(); }
Expected valueToTest.Foo not to be <null>. Expected valueToTest.Baz to be <null>, but found System.Object (HashCode=35410979).
/// <summary> /// Assertions that test for null values /// </summary> [Test] public void NullChecks() { var valueToTest = new { Foo = (object) null, Baz = new object() }; // Constraint-style asserts: valueToTest.Foo.ShouldBeNull(); valueToTest.Baz.ShouldNotBeNull(); }
Shouldly.ShouldAssertException : valueToTest.Baz should be null but was System.Object (35567111) Shouldly.ShouldAssertException : valueToTest.Foo should not be null but was
This next set will compare values relative to each other (greater than, less than, etc).
A benefit of using NUnit
This is an area where I think NUnit and Fluent Assertions really outshines the other frameworks. With the other frameworks, you're mostly limited to Assert.IsTrue(some_condition)
, which only gives you a pass/fail result. With NUnit's Constraint model and Fluent Assertions model, you get a much richer syntax that provides significantly more detail in the built-in failure message.
Note that you can take it even further with NUnit and not just compare two values, but provide a margin of error using the .Within()
helpers. This is particularly helpful when comparing floating-point numbers where the actual value may suffer from slight variations from your expected value.
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [Test] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; int zero = 0; double notANumber = double.NaN; // NaN == 0D / 0D double infinity = double.PositiveInfinity; bool trueValue = true; bool falseValue = false; DateTime jan1 = new DateTime(2019, 01, 01); Assert.That(bigNumber, Is.GreaterThan(smallNumber)); Assert.That(bigNumber, Is.GreaterThanOrEqualTo(smallNumber)); Assert.That(smallNumber, Is.LessThan(bigNumber)); Assert.That(smallNumber, Is.LessThanOrEqualTo(bigNumber)); Assert.That(trueValue, Is.True); Assert.That(falseValue, Is.False); Assert.That(bigNumber, Is.Positive); Assert.That(bigNumber, Is.Not.Negative); Assert.That(smallNumber, Is.Negative); Assert.That(smallNumber, Is.Not.Positive); Assert.That(zero, Is.Zero); Assert.That(bigNumber, Is.Not.Zero); Assert.That(notANumber, Is.NaN); Assert.That(infinity, Is.Not.NaN); Assert.That(zero, Is.InRange(-100, 5)); Assert.That(zero, Is.Not.InRange(1, 10)); Assert.That(jan1, Is.InRange(new DateTime(2018, 01, 01), new DateTime(2019, 12, 31))); Assert.That(zero, Is.AnyOf(42, 0, 100)); Assert.That(2.333333d, Is.EqualTo(2.3).Within(0.5)); Assert.That(jan1, Is.EqualTo(new DateTime(2019, 01, 10)).Within(10).Days); Assert.That(95, Is.EqualTo(100).Within(8).Percent); }
Expected: not greater than or equal to -2147483648 But was: 2147483647 Expected: less than 2147483647 But was: 2147483647 Expected: False But was: True Expected: 0 But was: 2147483647 Expected: NaN But was: ∞ Expected: in range (1,10) But was: 0 Expected: not in range (1/1/2018 12:00:00 AM,12/31/2019 12:00:00 AM) But was: 2019-01-01 00:00:00 Expected: any of < 42, 0, 100 > But was: 33 Expected: 5.2999999999999998d +/- 0.5d But was: 2.3333330000000001d Expected: not equal to 2019-01-10 00:00:00 +/- 10.00:00:00 But was: 2019-01-01 00:00:00 Expected: 100 +/- 8 Percent But was: 76
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [Test] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; int zero = 0; double notANumber = double.NaN; // NaN == 0D / 0D double infinity = double.PositiveInfinity; bool trueValue = true; bool falseValue = false; Assert.Greater(bigNumber, smallNumber); Assert.GreaterOrEqual(bigNumber, smallNumber); Assert.Less(smallNumber, bigNumber); Assert.LessOrEqual(smallNumber, bigNumber); Assert.True(trueValue); Assert.False(falseValue); Assert.Positive(bigNumber); Assert.Negative(smallNumber); Assert.Zero(zero); Assert.NotZero(bigNumber); Assert.IsNaN(notANumber); }
Expected: greater than or equal to 2147483647 But was: -2147483648 Expected: less than -2147483648 But was: 2147483647 Expected: False But was: True Expected: 0 But was: 2147483647 Expected: NaN But was: ∞
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [Fact] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; int zero = 0; double notANumber = double.NaN; // NaN == 0D / 0D double infinity = double.PositiveInfinity; bool trueValue = true; bool falseValue = false; DateTime jan1 = new DateTime(2019, 01, 01); // Constraint-style asserts: Assert.True(bigNumber > smallNumber); Assert.True(bigNumber >= smallNumber); Assert.True(smallNumber < bigNumber); Assert.True(smallNumber <= bigNumber); Assert.True(trueValue); Assert.False(falseValue); Assert.InRange(zero, -100, 5); Assert.NotInRange(zero, 1, 10); Assert.InRange(jan1, new DateTime(2018, 01, 01), new DateTime(2019, 12, 31)); }
Xunit.Sdk.TrueException: Assert.True() Failure Expected: True Actual: False Xunit.Sdk.InRangeException: Assert.InRange() Failure Range: (-100 - 5) Actual: 23 Xunit.Sdk.InRangeException: Assert.InRange() Failure Range: (1/1/2018 12:00:00 AM - 12/31/2018 12:00:00 AM) Actual: 1/1/2019 12:00:00 AM
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [TestMethod] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; bool trueValue = true; bool falseValue = false; Assert.IsTrue(bigNumber > smallNumber); Assert.IsTrue(bigNumber >= smallNumber); Assert.IsTrue(smallNumber < bigNumber); Assert.IsTrue(smallNumber <= bigNumber); Assert.IsTrue(trueValue); Assert.IsFalse(falseValue); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsTrue failed. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsFalse failed.
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [Test] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; int zero = 0; bool trueValue = true; bool falseValue = false; DateTime jan1 = new DateTime(2019, 01, 01); bigNumber.Should().BeGreaterThan(smallNumber); bigNumber.Should().BeGreaterOrEqualTo(smallNumber); smallNumber.Should().BeLessThan(bigNumber); smallNumber.Should().BeLessOrEqualTo(bigNumber); trueValue.Should().BeTrue(); falseValue.Should().BeFalse(); bigNumber.Should().BePositive(); smallNumber.Should().BeNegative(); zero.Should().BeInRange(-100, 5); zero.Should().NotBeInRange(1, 10); jan1.Should().BeAfter(new DateTime(2018, 01, 01)).And.BeBefore(new DateTime(2019, 12, 31)); zero.Should().BeOneOf(42, 0, 100); 2.333333d.Should().BeApproximately(2.3, 0.5); jan1.Should().BeCloseTo(new DateTime(2019, 01, 10), 10.Days()); }
Expected bigNumber to be less than -2147483648, but found 2147483647. Expected smallNumber to be greater or equal to 2147483647, but found -2147483648. Expected trueValue to be false, but found True. Expected smallNumber to be positive, but found -2147483648. Expected zero to be between 1 and 10, but found 0. Expected jan1 to be after <2018-01-01>, but found <2017-01-01>. Expected zero to be one of {42, 0, 100}, but found 33. Expected floatNum to approximate 5.3 +/- 0.5, but 2.333333 differed by 2.9666669999999997. Expected jan1 to be within 10d from <2019-01-10>, but found <2019-11-01>.
/// <summary> /// Assertions that compare the value to a set of constraints /// </summary> [Test] public void ComparisonChecks() { int bigNumber = int.MaxValue; int smallNumber = int.MinValue; int zero = 0; bool trueValue = true; bool falseValue = false; DateTime jan1 = new DateTime(2019, 01, 01); bigNumber.ShouldBeGreaterThan(smallNumber); bigNumber.ShouldBeGreaterThanOrEqualTo(smallNumber); smallNumber.ShouldBeLessThan(bigNumber); smallNumber.ShouldBeLessThanOrEqualTo(bigNumber); trueValue.ShouldBeTrue(); falseValue.ShouldBeFalse(); bigNumber.ShouldBePositive(); smallNumber.ShouldBeNegative(); zero.ShouldBeInRange(-100, 5); zero.ShouldNotBeInRange(1, 10); jan1.ShouldBeInRange(new DateTime(2018, 01, 01), new DateTime(2019, 12, 31)); zero.ShouldBeOneOf(42, 0, 100); 2.333333d.ShouldBe(2.3, 0.5); jan1.ShouldBe(new DateTime(2019, 01, 10), TimeSpan.FromDays(10)); }
Shouldly.ShouldAssertException : bigNumber should be less than -2147483648 but was 2147483647 Shouldly.ShouldAssertException : trueValue should be False but was True Shouldly.ShouldAssertException : zero should be in range { from = 1, to = 10 } but was 0 Shouldly.ShouldAssertException : jan1 should not be in range { from = 1/1/2018 12:00:00 AM, to = 12/31/2019 12:00:00 AM } but was 2019-01-01T00:00:00.0000000 Shouldly.ShouldAssertException : zero should be one of [42, 0, 100] but was 33 Shouldly.ShouldAssertException : myNum should be within 0.5d of 2.3d but was 1.2222222d Shouldly.ShouldAssertException : jan1 should not be within 10.00:00:00 of 2019-01-10T00:00:00.0000000 but was 2019-01-01T00:00:00.0000000
String Checks
Verify the contents of a string, from simple patters (StartsWith, Contains) to more complicated Regular Expression matching.
/// <summary> /// String-specific checks /// </summary> [Test] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; Assert.That("", Is.Empty); Assert.That(valueToTest, Is.Not.Empty); Assert.That(valueToTest, Does.Contain("Bar")); Assert.That(valueToTest, Does.Not.Contain("Bang")); Assert.That(valueToTest, Does.StartWith("Foo")); Assert.That(valueToTest, Does.Not.StartWith("Bar")); Assert.That(valueToTest, Does.EndWith("Bin")); Assert.That(valueToTest, Does.Not.EndWith("Baz")); Assert.That(valueToTest, Is.EqualTo("foo bar baz bin").IgnoreCase); Assert.That(valueToTest, Is.Not.EqualTo("something else").IgnoreCase); Assert.That(valueToTest, Does.Match("^Foo.*Bin$")); // param is a regex pattern Assert.That(valueToTest, Does.Not.Match("^Foo.*Bar$")); // param is a regex pattern }
Expected: <empty> But was: "asdf" Expected: <empty> But was: "Foo Bar Baz Bin" Expected: not String containing "Bar" But was: "Foo Bar Baz Bin" Expected: String containing "Bang" But was: "Foo Bar Baz Bin" Expected: not String starting with "Foo" But was: "Foo Bar Baz Bin" Expected: String starting with "Bar" But was: "Foo Bar Baz Bin" Expected: not String ending with "Bin" But was: "Foo Bar Baz Bin" Expected: String ending with "Baz" But was: "Foo Bar Baz Bin" Expected: not equal to "foo bar baz bin", ignoring case But was: "Foo Bar Baz Bin" Expected string length 14 but was 15. Strings differ at index 0. Expected: "something else", ignoring case But was: "Foo Bar Baz Bin" -----------^ Expected: not String matching "^Foo.*Bin$" But was: "Foo Bar Baz Bin" Expected: String matching "^Foo.*Bar$" But was: "Foo Bar Baz Bin"
/// <summary> /// String-specific checks /// </summary> [Test] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; StringAssert.Contains("Bar", valueToTest); StringAssert.DoesNotContain("Bang", valueToTest); StringAssert.StartsWith("Foo", valueToTest); StringAssert.DoesNotStartWith("Bar", valueToTest); StringAssert.EndsWith("Bin", valueToTest); StringAssert.DoesNotEndWith("Baz", valueToTest); StringAssert.AreEqualIgnoringCase("foo bar baz bin", valueToTest); StringAssert.AreNotEqualIgnoringCase("something else", valueToTest); StringAssert.IsMatch("^Foo.*Bin$", valueToTest); //first param is a regex pattern StringAssert.DoesNotMatch("^Foo.*Bar$", valueToTest); //first param is a regex pattern }
Expected: not String containing "Bar" But was: "Foo Bar Baz Bin" Expected: String containing "Bang" But was: "Foo Bar Baz Bin" Expected: not String starting with "Foo" But was: "Foo Bar Baz Bin" Expected: String starting with "Bar" But was: "Foo Bar Baz Bin" Expected: not String ending with "Bin" But was: "Foo Bar Baz Bin" Expected: String ending with "Baz" But was: "Foo Bar Baz Bin" Expected: not equal to "foo bar baz bin", ignoring case But was: "Foo Bar Baz Bin" Expected string length 14 but was 15. Strings differ at index 0. Expected: "something else", ignoring case But was: "Foo Bar Baz Bin" -----------^ Expected: not String matching "^Foo.*Bin$" But was: "Foo Bar Baz Bin" Expected: String matching "^Foo.*Bar$" But was: "Foo Bar Baz Bin"
/// <summary> /// String-specific checks /// </summary> [Fact] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; Assert.Contains("Bar", valueToTest); Assert.DoesNotContain("Bang", valueToTest); Assert.StartsWith("Foo", valueToTest); Assert.EndsWith("Bin", valueToTest); Assert.Equal("foo bar baz bin", valueToTest, ignoreCase: true); Assert.NotEqual("something else", valueToTest, StringComparer.InvariantCultureIgnoreCase); Assert.Matches("^Foo.*Bin$", valueToTest); //first param is a regex pattern Assert.Matches(new Regex("^Foo.*Bin$"), valueToTest); Assert.DoesNotMatch("^Foo.*Bar$", valueToTest); //first param is a regex pattern Assert.DoesNotMatch(new Regex("^Foo.*Bar$"), valueToTest); }
Xunit.Sdk.DoesNotContainException: Assert.DoesNotContain() Failure Found: Bar In value: Foo Bar Baz Bin Xunit.Sdk.ContainsException: Assert.Contains() Failure Not found: Bang In value: Foo Bar Baz Bin Xunit.Sdk.StartsWithException: Assert.StartsWith() Failure: Expected: Bin Actual: Foo... Xunit.Sdk.EndsWithException: Assert.EndsWith() Failure: Expected: Foo Actual: ···Bin Xunit.Sdk.EqualException: Assert.Equal() Failure ↓ (pos 0) Expected: something else Actual: Foo Bar Baz Bin ↑ (pos 0) Xunit.Sdk.NotEqualException: Assert.NotEqual() Failure Expected: Not "foo bar baz bin" Actual: "Foo Bar Baz Bin" Xunit.Sdk.MatchesException: Assert.Matches() Failure: Regex: ^Foo.*Bar$ Value: Foo Bar Baz Bin Xunit.Sdk.DoesNotMatchException: Assert.DoesNotMatch() Failure: Regex: ^Foo.*Bin Value: Foo Bar Baz Bin
/// <summary> /// String-specific checks /// </summary> [TestMethod] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; StringAssert.Contains(valueToTest, "Bar"); StringAssert.StartsWith(valueToTest, "Foo"); StringAssert.EndsWith(valueToTest, "Bin"); StringAssert.Matches(valueToTest, new Regex( "^Foo.*Bin$")); StringAssert.DoesNotMatch(valueToTest, new Regex( "^Foo.*Bar$")); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: StringAssert.Contains failed. String 'Foo Bar Baz Bin' does not contain string 'asdf'. . Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: StringAssert.StartsWith failed. String 'Foo Bar Baz Bin' does not start with string 'Bar'. . Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: StringAssert.EndsWith failed. String 'Foo Bar Baz Bin' does not end with string 'Far'. . Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: StringAssert.Matches failed. String 'Foo Bar Baz Bin' does not match pattern '^Foo.*Bar$'. . Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: StringAssert.DoesNotMatch failed. String 'Foo Bar Baz Bin' matches pattern '^Foo.*Bin$'. .
/// <summary> /// String-specific checks /// </summary> [Test] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; "".Should().BeEmpty(); valueToTest.Should().NotBeEmpty(); valueToTest.Should().Contain("Bar"); valueToTest.Should().NotContain("Bang"); valueToTest.Should().StartWith("Foo"); valueToTest.Should().NotStartWith("Bar"); valueToTest.Should().EndWith("Bin"); valueToTest.Should().NotEndWith("Baz"); valueToTest.Should().BeEquivalentTo("foo bar baz bin"); valueToTest.Should().NotBeEquivalentTo("something else"); valueToTest.Should().MatchRegex("^Foo.*Bin$"); // param is a regex pattern valueToTest.Should().NotMatchRegex("^Foo.*Bar$"); // param is a regex pattern valueToTest.Should().Match("Foo*Bin"); // param is a wildcard pattern valueToTest.Should().NotMatch("Foo*Bar"); // param is a wildcard pattern }
Expected string to be empty, but found "asdf". Did not expect valueToTest to be empty. Did not expect valueToTest "Foo Bar Baz Bin" to contain "Bar". Expected valueToTest "Foo Bar Baz Bin" to contain "Bang". Expected valueToTest that does not start with "Foo", but found "Foo Bar Baz Bin". Expected valueToTest to start with "Bar", but "Foo Bar Baz Bin" differs near "Foo" (index 0). Expected valueToTest "Foo Bar Baz Bin" not to end with "Bin". Expected valueToTest "Foo Bar Baz Bin" to end with "Baz". Expected valueToTest not to be equivalent to "foo bar baz bin", but they are. Expected valueToTest to be equivalent to "something else" with a length of 14, but "Foo Bar Baz Bin" has a length of 15, differs near "Foo" (index 0). Did not expect valueToTest to match regex "^Foo.*Bin$", but "Foo Bar Baz Bin" matches. Expected valueToTest to match "Foo*Bar", but "Foo Bar Baz Bin" does not.
/// <summary> /// String-specific checks /// </summary> [Test] public void StringChecks() { var valueToTest = "Foo Bar Baz Bin"; string.Empty.ShouldBeEmpty(); valueToTest.ShouldNotBeEmpty(); valueToTest.ShouldContain("Bar"); valueToTest.ShouldNotContain("Bang"); valueToTest.ShouldStartWith("Foo"); valueToTest.ShouldNotStartWith("Bar"); valueToTest.ShouldEndWith("Bin"); valueToTest.ShouldNotEndWith("Baz"); valueToTest.ShouldMatch("^Foo.*Bin$"); // param is a regex pattern valueToTest.ShouldNotMatch("^Foo.*Bar$"); // param is a regex pattern }
Shouldly.ShouldAssertException : valueToTest should be empty but was "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should not be empty but was Shouldly.ShouldAssertException : valueToTest should not contain (case insensitive comparison) "Bar" but was actually "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should contain (case insensitive comparison) "Bang" but was actually "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should not start with "Foo" but was "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should start with "Bar" but was "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should not end with "Bin" but was "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should end with "Baz" but was "Foo Bar Baz Bin" Shouldly.ShouldAssertException : valueToTest should not match "^Foo.*Bin$" but did Shouldly.ShouldAssertException : valueToTest should match "^Foo.*Bar$" but was "Foo Bar Baz Bin"
Type Checks
Verify whether an object is a certain type, or could be used as a certain type.
/// <summary> /// Tests related to object types and inheritance /// </summary> [Test] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] { }; Assert.That("foo", Is.AssignableFrom(typeof(string))); Assert.That("foo", Is.AssignableFrom<string>()); Assert.That("foo", Is.Not.AssignableFrom(typeof(int))); Assert.That("foo", Is.Not.AssignableFrom<int>()); Assert.That(stringList, Is.InstanceOf(typeof(List<string>))); Assert.That(stringList, Is.InstanceOf<List<string>>()); Assert.That(intEnumerable, Is.Not.InstanceOf(typeof(List<int>))); Assert.That(intEnumerable, Is.Not.InstanceOf<List<int>>()); Assert.That(stringList, Is.AssignableTo(typeof(IEnumerable<string>))); Assert.That(stringList, Is.AssignableTo<IEnumerable<string>>()); Assert.That(stringList, Is.Not.AssignableTo(typeof(string[]))); Assert.That(stringList, Is.Not.AssignableTo<string[]>()); Assert.That(intEnumerable, Is.TypeOf(typeof(int[]))); //must be exact type Assert.That(intEnumerable, Is.TypeOf<int[]>()); //must be exact type Assert.That(stringList, Is.Not.TypeOf(typeof(IEnumerable<string>))); //must be exact type Assert.That(stringList, Is.Not.TypeOf<IEnumerable<string>>()); //must be exact type }
Expected: not assignable from <System.String> But was: <System.String> Expected: assignable from <System.Int32> But was: <System.String> Expected: not instance of <System.Collections.Generic.List`1[System.String]> But was: <System.Collections.Generic.List`1[System.String]> Expected: instance of <System.Collections.Generic.List`1[System.Int32]> But was: <System.Int32[]> Expected: not assignable to <System.Collections.Generic.IEnumerable`1[System.String]> But was: <System.Collections.Generic.List`1[System.String]> Expected: assignable to <System.String[]> But was: <System.Collections.Generic.List`1[System.String]> Expected: not <System.Int32[]> But was: <System.Int32[]> Expected: <System.Collections.Generic.IEnumerable`1[System.String]> But was: <System.Collections.Generic.List`1[System.String]>
/// <summary> /// Tests related to object types and inheritance /// </summary> [Test] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] { }; Assert.IsAssignableFrom(typeof(string), "foo"); Assert.IsAssignableFrom<string>("foo"); Assert.IsNotAssignableFrom(typeof(int), "foo"); Assert.IsNotAssignableFrom<int>("foo"); Assert.IsInstanceOf(typeof(List<string>), stringList); Assert.IsInstanceOf<List<string>>(stringList); Assert.IsNotInstanceOf(typeof(List<int>), intEnumerable); Assert.IsNotInstanceOf<List<int>>(intEnumerable); }
Expected: not assignable from <System.String> But was: <System.String> Expected: assignable from <System.Int32> But was: <System.String> Expected: not instance of <System.Collections.Generic.List`1[System.String]> But was: <System.Collections.Generic.List`1[System.String]> Expected: instance of <System.Collections.Generic.List`1[System.Int32]> But was: <System.Int32[]>
/// <summary> /// Tests related to object types and inheritance /// </summary> [Fact] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] { }; Assert.IsAssignableFrom<string>("foo"); Assert.IsType<List<string>>(stringList); Assert.IsNotType<List<int>>(intEnumerable); }
Xunit.Sdk.IsAssignableFromException: Assert.IsAssignableFrom() Failure Expected: typeof(int) Actual: typeof(string) Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Collections.Generic.List`1[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] Actual: System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] Xunit.Sdk.IsNotTypeException: Assert.IsNotType() Failure Expected: typeof(System.Collections.Generic.List<string>) Actual: typeof(System.Collections.Generic.List<string>)
/// <summary> /// Tests related to object types and inheritance /// </summary> [TestMethod] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] { }; Assert.IsInstanceOfType(stringList, typeof(List<string>)); Assert.IsNotInstanceOfType(intEnumerable, typeof(List<int>)); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsNotInstanceOfType failed. Wrong Type:<System.Collections.Generic.List`1[System.String]>. Actual type:<System.Collections.Generic.List`1[System.String]>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.IsInstanceOfType failed. Expected type:<System.Collections.Generic.List`1[System.Int32]>. Actual type:<System.Int32[]>.
/// <summary> /// Tests related to object types and inheritance /// </summary> [Test] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] { }; stringList.Should().BeOfType(typeof(List<string>)); stringList.Should().BeOfType<List<string>>(); intEnumerable.Should().NotBeOfType(typeof(List<int>)); intEnumerable.Should().NotBeOfType< List<int>>(); stringList.Should().BeAssignableTo(typeof(IEnumerable<string>)); stringList.Should().BeAssignableTo<IEnumerable<string>>(); stringList.Should().NotBeAssignableTo(typeof(string[])); stringList.Should().NotBeAssignableTo<string[]>(); }
Expected type not to be [System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089], but it is. Expected type to be System.Collections.Generic.List`1[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], but found System.Int32[]. Expected stringList to not be assignable to System.Collections.Generic.IEnumerable`1[System.String], but System.Collections.Generic.List`1[System.String] is. Expected stringList to be assignable to System.String[], but System.Collections.Generic.List`1[System.String] is not.
/// <summary> /// Tests related to object types and inheritance /// </summary> [Test] public void TypeChecks() { IList<string> stringList = new List<string>(); IEnumerable<int> intEnumerable = new int[] {1, 2, 3}; stringList.ShouldBeAssignableTo(typeof(IEnumerable<string>)); stringList.ShouldBeAssignableTo<IEnumerable<string>>(); stringList.ShouldNotBeAssignableTo(typeof(string[])); stringList.ShouldNotBeAssignableTo<string[]>(); intEnumerable.ShouldBeOfType(typeof(int[])); //must be exact type intEnumerable.ShouldBeOfType<int[]>(); //must be exact type stringList.ShouldNotBeOfType(typeof(IEnumerable<string>)); //must be exact type stringList.ShouldNotBeOfType<IEnumerable<string>>(); //must be exact type }
Shouldly.ShouldAssertException : stringList should not be assignable to System.Collections.Generic.IEnumerable`1[System.String] but was [] (System.Collections.Generic.List`1[System.String]) Shouldly.ShouldAssertException : stringList should be assignable to System.String[] but was System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] Shouldly.ShouldAssertException : intEnumerable should not be of type System.Int32[] but was [1, 2, 3] Shouldly.ShouldAssertException : stringList should be of type System.Collections.Generic.IEnumerable`1[System.String] but was System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
Collection Checks
Verify the contents of a collection meet some expectations.
/// <summary> /// Checks specific to collections /// </summary> [Test] public void CollectionChecks() { var objArr = new object[] {new object(), 42, "my string"}; var stringArr = new object[] {"foo", "bar", "baz", "bin", ""}; var intList = Enumerable.Range(0, 100); Assert.That(stringArr, Is.All.TypeOf<string>()); Assert.That(intList, Is.All.GreaterThanOrEqualTo(0)); Assert.That(objArr, Is.All.Not.Null); Assert.That(intList, Is.Unique); Assert.That(intList, Is.EqualTo(Enumerable.Range(0, 100))); Assert.That(intList, Is.Not.EqualTo(Enumerable.Range(1, 5))); Assert.That(stringArr, Is.EquivalentTo(new string[] { "bar", "baz", "", "bin", "foo" })); Assert.That(stringArr, Is.Not.EquivalentTo(new string[] { "bar", "baz" })); Assert.That(stringArr, Has.Member("foo")); Assert.That(stringArr, Does.Contain("foo")); Assert.That(stringArr, Contains.Item("foo")); Assert.That(stringArr, Has.No.Member("zoom")); Assert.That(stringArr, Does.Not.Contain("zoom")); Assert.That(Enumerable.Range(5, 20), Is.SubsetOf(intList)); Assert.That(Enumerable.Range(-1, 1), Is.Not.SubsetOf(intList)); Assert.That(intList, Is.SupersetOf(Enumerable.Range(5, 20))); Assert.That(intList, Is.Not.SupersetOf(Enumerable.Range(-1, 1))); Assert.That(new int[] { }, Is.Empty); Assert.That(intList, Is.Not.Empty); Assert.That(new int[] { 1, 2, 3 }, Is.Ordered); Assert.That(new int[] { 2, 1, 3 }, Is.Not.Ordered); string[] sarray = new string[] { "a", "aa", "aaa" }; Assert.That(sarray, Is.Ordered.By("Length")); Assert.That(sarray, Is.Ordered.Ascending.By("Length")); Assert.That(intList, Has.Exactly(100).Items); Assert.That(intList, Has.Exactly(50).Items.GreaterThanOrEqualTo(50)); Assert.That(intList, Has.None.LessThan(0)); Assert.That(objArr, Has.Some.TypeOf<string>()); Assert.That(intList, Has.All.GreaterThanOrEqualTo(0)); }
Expected: all items <System.Int32> But was: < "foo", "bar", "baz", "bin", <string.Empty> > First non-matching item at index [0]: "foo" Expected: all items greater than or equal to 3 But was: < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > First non-matching item at index [0]: 0 Expected: all items unique But was: < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > Expected: equivalent to < "bar", "baz", <string.Empty>, "bin", "foo", "extra" > But was: < "foo", "bar", "baz", "bin", <string.Empty> > Missing (1): < "extra" > Expected: some item equal to "extra" But was: < "foo", "bar", "baz", "bin", <string.Empty> > Expected: subset of < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > But was: < -5, -4, -3, -2, -1, 0, 1, 2, 3, 4... > Expected: superset of < -5, -4, -3, -2, -1, 0, 1, 2, 3, 4... > But was: < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > Expected: <empty> But was: < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > Expected: collection ordered But was: < 2, 1, 3 > Ordering breaks at index [1]: 1 Expected: collection ordered by "Length" But was: < "aa", "a", "aaa" > Ordering breaks at index [1]: "a" Expected: exactly 50 items greater than or equal to 51 But was: 49 items < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... >
/// <summary> /// Checks specific to collections /// </summary> [Test] public void CollectionChecks() { var objArr = new object[] {new object(), 42, "my string"}; var stringArr = new object[] {"foo", "bar", "baz", "bin", ""}; var intList = Enumerable.Range(0, 100); CollectionAssert.AllItemsAreInstancesOfType(stringArr, typeof(string)); CollectionAssert.AllItemsAreNotNull(objArr); CollectionAssert.AllItemsAreUnique(intList); CollectionAssert.AreEqual(Enumerable.Range(0, 100), intList); CollectionAssert.AreNotEqual(Enumerable.Range(1, 5), intList); CollectionAssert.AreEquivalent(new string[] { "bar", "baz", "", "bin", "foo" }, stringArr); CollectionAssert.AreNotEquivalent(new string[] { "bar", "baz" }, stringArr); CollectionAssert.Contains(stringArr, "foo"); CollectionAssert.DoesNotContain(stringArr, "zoom"); CollectionAssert.IsSubsetOf(Enumerable.Range(5, 20), intList); CollectionAssert.IsNotSubsetOf(Enumerable.Range(-1, 1), intList); CollectionAssert.IsEmpty(new int[] { }); CollectionAssert.IsNotEmpty(new int[] { 1, 2 }); CollectionAssert.IsOrdered(new int[] { 1, 2, 3 }); }
Expected: all items unique But was: < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > Expected and actual are both <System.Linq.Enumerable+<RangeIterator>d__113> Values differ at index [100] Expected: equivalent to < "bar", "baz", <string.Empty>, "bin", "foo", "extra" > But was: < "foo", "bar", "baz", "bin", <string.Empty> > Missing (1): < "extra" > Expected: some item equal to "extra" But was: < "foo", "bar", "baz", "bin", <string.Empty> > Expected: subset of < 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... > But was: < -5, -4, -3, -2, -1, 0, 1, 2, 3, 4... > Expected: <empty> But was: < 1, 2 > Expected: collection ordered But was: < 2, 1, 3 > Ordering breaks at index [1]: 1
/// <summary> /// Checks specific to collections /// </summary> [Fact] public void CollectionChecks() { var objArr = new object[] { new object(), 42, "my string" }; var stringArr = new string[] { "foo", "bar", "baz", "bin", "" }; var intList = Enumerable.Range(0, 100); Assert.All(stringArr, s => Assert.IsType<string>(s)); Assert.All(objArr, Assert.NotNull ); Assert.Equal(Enumerable.Range(0, 100), intList); Assert.NotEqual(Enumerable.Range(1, 5), intList); Assert.Contains("foo", stringArr); Assert.DoesNotContain("zoom", stringArr); Assert.Subset(intList.ToHashSet(), Enumerable.Range(5, 20).ToHashSet()); Assert.Superset(Enumerable.Range(5, 20).ToHashSet(), intList.ToHashSet()); Assert.Empty(new int[] { }); Assert.NotEmpty(new int[] { 1, 2 }); }
Xunit.Sdk.AllException: Assert.All() Failure: 5 out of 5 items in the collection did not pass. [4]: Item: Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Int32 Actual: System.String ... [3]: Item: bin Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Int32 Actual: System.String ... [2]: Item: baz Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Int32 Actual: System.String ... [1]: Item: bar Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Int32 Actual: System.String ... [0]: Item: foo Xunit.Sdk.IsTypeException: Assert.IsType() Failure Expected: System.Int32 Actual: System.String Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: <RangeIterator>d__113 [1, 2, 3, 4, 5, ...] Actual: <RangeIterator>d__113 [0, 1, 2, 3, 4, ...] Xunit.Sdk.ContainsException: Assert.Contains() Failure Not found: extra In value: String[] ["foo", "bar", "baz", "bin", ""] Xunit.Sdk.SubsetException: Assert.Subset() Failure Expected: HashSet<Int32> [0, 1, 2, 3, 4, ...] Actual: HashSet<Int32> [-5, -4, -3, -2, -1, ...] Xunit.Sdk.SupersetException: Assert.Superset() Failure Expected: HashSet<Int32> [-5, -4, -3, -2, -1, ...] Actual: HashSet<Int32> [0, 1, 2, 3, 4, ...] Xunit.Sdk.EmptyException: Assert.Empty() Failure Collection: [1, 2]
/// <summary> /// Checks specific to collections /// </summary> [TestMethod] public void CollectionChecks() { var objArr = new object[] { new object(), 42, "my string" }; var stringArr = new object[] { "foo", "bar", "baz", "bin", "" }; var intList = Enumerable.Range(0, 100).ToList(); CollectionAssert.AllItemsAreInstancesOfType(stringArr, typeof(string)); CollectionAssert.AllItemsAreNotNull(objArr); CollectionAssert.AllItemsAreUnique(intList); CollectionAssert.AreEqual(Enumerable.Range(0, 100).ToList(), intList); CollectionAssert.AreNotEqual(Enumerable.Range(1, 5).ToList(), intList); CollectionAssert.AreEquivalent(new string[] { "bar", "baz", "", "bin", "foo" }, stringArr); CollectionAssert.AreNotEquivalent(new string[] { "bar", "baz" }, stringArr); CollectionAssert.Contains(stringArr, "foo"); CollectionAssert.DoesNotContain(stringArr, "zoom"); CollectionAssert.IsSubsetOf(Enumerable.Range(5, 20).ToList(), intList); CollectionAssert.IsNotSubsetOf(Enumerable.Range(-1, 1).ToList(), intList); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.AllItemsAreInstancesOfType failed. Element at index 0 is not of expected type. Expected type:<System.Int32>. Actual type:<System.String>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.AllItemsAreUnique failed. Duplicate item found:<10>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.AreEqual failed. (Different number of elements.) Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.AreEquivalent failed. The number of elements in the collections do not match. Expected:<6>. Actual:<5>. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.Contains failed. Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: CollectionAssert.IsSubsetOf failed.
/// <summary> /// Checks specific to collections /// </summary> [Test] public void CollectionChecks() { var objArr = new object[] {new object(), 42, "my string"}; var stringArr = new object[] {"foo", "bar", "baz", "bin", ""}; var intList = Enumerable.Range(0, 100); stringArr.Should().ContainItemsAssignableTo<string>(); intList.Should().Contain(x => x >= 0); objArr.Should().NotContainNulls(); intList.Should().OnlyHaveUniqueItems(); intList.Should().Equal(Enumerable.Range(0, 100)); intList.Should().NotEqual(Enumerable.Range(1, 5)); stringArr.Should().BeEquivalentTo(new string[] { "bar", "baz", "", "bin", "foo" }); stringArr.Should().NotBeEquivalentTo(new string[] { "bar", "baz" }); stringArr.Should().Contain("foo"); stringArr.Should().NotContain("zoom"); Enumerable.Range(5, 20).Should().BeSubsetOf(intList); Enumerable.Range(-1, 1).Should().NotBeSubsetOf(intList); new int[] { }.Should().BeEmpty(); intList.Should().NotBeEmpty(); new int[] { 1, 2, 3 }.Should().BeInAscendingOrder(); new int[] { 2, 1, 3 }.Should().NotBeInDescendingOrder(); string[] sarray = new string[] { "a", "aa", "aaa" }; sarray.Should().BeInAscendingOrder(new StringLengthComparer()); intList.Should().HaveCount(100); intList.Should().OnlyContain(x => x >= 0); } private class StringLengthComparer : IComparer<object> { public int Compare(object x, object y) { if (x == null || y == null) { if (x == y) return 0; if (x == null) return -1; else return 1; } if (x is string xs && y is string ys) { return xs.Length.CompareTo(ys.Length); } else { return -1; } } }
Expected stringArr to contain only items of type System.Int32, but item "foo" at index 0 is of type System.String. Expected intList to contain only items matching (x >= 3), but {0, 1, 2} do(es) not match. Expected objArr not to contain <null>s, but found one at index 3. Expected intList to only have unique items, but items {0, 1, 3} are not unique. Expected stringArr to be a collection with 5 item(s), but {"foo", "bar", "baz", "bin", "", "extra"} contains 1 item(s) more than {"bar", "baz", "", "bin", "foo"}. Expected stringArr {"foo", "bar", "baz", "bin", ""} not be equivalent with collection {"bar", "baz", "", "bin", "foo"}. Expected stringArr {"foo", "bar", "baz", "bin", ""} to contain "extra". Expected Enumerable.Range(-5, 20) to be a subset of {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, …68 more…}, but items {-5, -4, -3, -2, -1} are not part of the superset. Expected collection to be empty, but found {1, 2, 3}. Expected collection to contain items in descending order, but found {2, 1, 3} where item at index 0 is in wrong order. Expected sarray to contain items in ascending order, but found {"a", "aaa", "aa"} where item at index 1 is in wrong order. Expected intList to contain 50 item(s), but found 100.
/// <summary> /// Checks specific to collections /// </summary> [Test] public void CollectionChecks() { var objArr = new object[] {new object(), 42, "my string"}; var stringArr = new object[] {"foo", "bar", "baz", "bin", ""}; var intList = Enumerable.Range(0, 100); stringArr.ShouldAllBe(x => x is string); intList.ShouldAllBe(x => x >= 0); objArr.ShouldAllBe(x => x != null); intList.ShouldBeUnique(); intList.ShouldBe(Enumerable.Range(0, 100)); intList.ShouldNotBe(Enumerable.Range(1, 5)); stringArr.ShouldBe(new string[] { "bar", "baz", "", "bin", "foo" }, true); stringArr.ShouldContain("foo"); stringArr.ShouldNotContain("zoom"); Enumerable.Range(5, 20).ShouldBeSubsetOf(intList); new int[] { }.ShouldBeEmpty(); intList.ShouldNotBeEmpty(); var intArr = new int[] { 1, 2, 3 }; intArr.ShouldBeInOrder(); string[] sarray = new string[] { "a", "aa", "aaa" }; sarray.ShouldBeInOrder(SortDirection.Ascending, new StringLengthComparer()); intList.Count().ShouldBe(100); intList.ShouldAllBe(x => x >= 0); } private class StringLengthComparer: IComparer<string> { public int Compare(string x, string y) { if (x == null || y == null) { if (x == y) return 0; if (x == null) return -1; else return 1; } return x.Length.CompareTo(y.Length); } }
Shouldly.ShouldAssertException : stringArr should satisfy the condition (x Is Int32) but ["foo", "bar", "baz", "bin", ""] do not Shouldly.ShouldAssertException : intList should satisfy the condition (x >= 3) but [0, 1, 2] do not Shouldly.ShouldAssertException : intList should be unique but [1] was duplicated Shouldly.ShouldAssertException : stringArr should be (ignoring order) ["bar", "baz", "", "bin", "foo"] but ["bar", "baz", "", "bin", "foo"] is missing ["extra"] Shouldly.ShouldAssertException : stringArr should contain "extra" but was actually ["foo", "bar", "baz", "bin", ""] Shouldly.ShouldAssertException : myList should be subset of [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99] but [-5, -4, -3, -2, -1] are outside subset Shouldly.ShouldAssertException : intList should be empty but had 100 items and was [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99] Shouldly.ShouldAssertException : intArr should be in ascending order but was not. The first out-of-order item was found at index 1: 1 Shouldly.ShouldAssertException : sarray should be in ascending order but was not. The first out-of-order item was found at index 2: aa Shouldly.ShouldAssertException : intList.Count() should be 50 but was 100
Exception Checks
If your code throws an exception, then it automatically fails the test (that's how Asserts work afterall). But sometimes you want to be deterministic in your tests. So these Exception Asserts allow you to do more detailed validation of the Exception you expect to be thrown.
/// <summary> /// Exception-specific checks /// </summary> [Test] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } Assert.That(() => { return; }, Throws.Nothing); Assert.That(MethodThatThrows, Throws.ArgumentException); Assert.That(MethodThatThrows, Throws.TypeOf<ArgumentException>()); Assert.That(() => throw new Exception("message"), Throws.InstanceOf<Exception>() .And.With.Property(nameof(Exception.Message)).EqualTo("message")); // Require an ApplicationException - derived types fail! Assert.That(() => throw new ApplicationException("message"), Throws.TypeOf<ApplicationException>()); // Allow both ApplicationException and any derived type Assert.That(() => throw new ApplicationException("message"), Throws.InstanceOf<Exception>()); }
Expected: No Exception to be thrown But was: <System.NotImplementedException: opps! Expected: <System.ArgumentException> But was: <System.NotImplementedException: opps! Expected: instance of <System.Exception> and property Message equal to "message" But was: "opps!" Expected: instance of <System.NotImplementedException> But was: <System.ApplicationException: message
/// <summary> /// Exception-specific checks /// </summary> [Test] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } Assert.DoesNotThrow(() => { return; }); Assert.Throws<ArgumentException>(MethodThatThrows); Assert.Throws<ArgumentException>( () => throw new ArgumentException()); Exception ex = Assert.Throws<Exception>(() => throw new Exception("message")); Assert.That(ex.Message, Is.EqualTo("message")); Assert.Throws(Is.TypeOf<Exception>().And.Message.EqualTo("message"), () => throw new Exception("message")); // Require an ApplicationException - derived types fail! Assert.Throws<ApplicationException>(() => throw new ApplicationException("message")); // Allow both ApplicationException and any derived type Assert.Throws(Is.InstanceOf<Exception>(), () => throw new ApplicationException("message")); }
Expected: No Exception to be thrown But was: <System.NotImplementedException: opps! Expected: <System.ArgumentException> But was: <System.NotImplementedException: opps! Expected: <System.Exception> But was: <System.NotImplementedException: message Expected: <System.NotImplementedException> and property Message equal to "message" But was: <System.Exception: opps! Expected: instance of <System.NotImplementedException> But was: <System.ApplicationException: message
/// <summary> /// Exception-specific checks /// </summary> [Fact] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } Assert.Throws<ArgumentException>(() => MethodThatThrows()); Exception ex = Assert.Throws<Exception>((Action)(() => throw new Exception("message"))); Assert.Equal("message", ex.Message); }
Xunit.Sdk.ThrowsException: Assert.Throws() Failure Expected: typeof(System.ArgumentException) Actual: typeof(System.NotImplementedException): opps!
/// <summary> /// Exception-specific checks /// </summary> [TestMethod] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } Assert.ThrowsException<ArgumentException>(() => MethodThatThrows()); Assert.ThrowsException<ArgumentException>(() => throw new ArgumentException()); Exception ex = Assert.ThrowsException<Exception>(() => throw new Exception("message")); Assert.AreEqual("message", ex.Message); }
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.ThrowsException failed. Threw exception NotImplementedException, but exception ArgumentException was expected. Exception Message: opps!
/// <summary> /// Exception-specific checks /// </summary> [Test] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } Action act = () => { return; }; act.Should().NotThrow(); act = () => MethodThatThrows(); act.Should().Throw<ArgumentException>(); act = () => throw new Exception("message"); act.Should().Throw<Exception>().And.Message.Should().Be("message"); // Require an ApplicationException - derived types fail! act = () => throw new ApplicationException("message"); act.Should().ThrowExactly<ApplicationException>(); }
Did not expect any exception, but found System.ArgumentException with message "Value does not fall within the expected range." Expected a <System.ArgumentException> to be thrown, but found <System.NotImplementedException>: " "System.NotImplementedException with message "The method or operation is not implemented." Expected act to be "message" with a length of 7, but "opps!" has a length of 5, differs near "opp" (index 0). Expected type to be System.NotImplementedException, but found System.ApplicationException.
/// <summary> /// Exception-specific checks /// </summary> [Test] public void ExceptionChecks() { void MethodThatThrows() { throw new ArgumentException(); } void MethodThatDoesNotThrow() { return;} Action actionThatThrows = MethodThatThrows; Action actionThatDoesNotThrow = MethodThatDoesNotThrow; actionThatThrows.ShouldThrow<ArgumentException>(); actionThatThrows.ShouldThrow(typeof(ArgumentException)); actionThatDoesNotThrow.ShouldNotThrow(); var ex = ((Action)(() => throw new Exception("message"))) .ShouldThrow<Exception>(); ex.Message.ShouldBe("message"); }
Shouldly.ShouldAssertException : `actionThatThrows()` should not throw but threw System.ArgumentException with message "Value does not fall within the expected range." Shouldly.ShouldAssertException : `actionThatDoesNotThrow()` should throw System.ArgumentException but did not Shouldly.ShouldAssertException : `actionThatThrows()` should throw System.NotImplementedException but threw System.ArgumentException Shouldly.ShouldAssertException : ex.Message should be "message" but was "different message" difference Difference | | | | | | | | | | | | | | | | | | \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ \|/ Index | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Expected Value | m e s s a g e Actual Value | d i f f e r e n t \s m e s s a g e Expected Code | 109 101 115 115 97 103 101 Actual Code | 100 105 102 102 101 114 101 110 116 32 109 101 115 115 97 103 101
Dynamically Set Test Results
This set of Asserts allow you to directly set the staus of a test as Pass, Fail, Ignored or Inconclusive (which isn't supported in most runners)
/// <summary> /// Assert calls that dynamically change the test results /// </summary> [Test] public void ForcedResults() { Assert.Pass("immediately end the test with a passing result"); Assert.Fail("immediately end the test with a failure result"); Assert.Ignore("dynamically cause a test or suite to be ignored at runtime"); Assert.Inconclusive("indicates that the test could not be completed with the data available"); }
/// <summary> /// Assert calls that dynamically change the test results /// </summary> [Fact] public void ForcedResults() { //There is no Assert.Fail option, but you can just thrown an exception. //Tip: Create your own Assert.Fail method to wrap throwing an exception throw new Exception("immediately end the test with a failure result"); //There's not a built-in Assert.Skip (aka Ignore), but there is a somewhat complicated //way to do it. Example code here: https://github.com/xunit/samples.xunit/tree/master/DynamicSkipExample }
/// <summary> /// Assert calls that dynamically change the test results /// </summary> [TestMethod] public void ForcedResults() { Assert.Inconclusive("indicates that the test could not be completed with the data available"); Assert.Fail("immediately end the test with a failure result"); }
Multiple Criteria Assert
The NUnit Constraint-style syntax, Shouldly, and Fluent Assertions all allow you to chain assert conditions together, so that a single Assert can match on multiple conditions at once.
/// <summary> /// Syntax for executing multiple assertions in the same test (ie: all asserts are run) /// </summary> [Test] public void MultipleCriteriaChecks() { object aNumber = 5.0; Assert.That(aNumber, Is.AssignableTo<int>().Or.AssignableTo<double>()); Assert.That(aNumber, Is.GreaterThanOrEqualTo(0).And.LessThanOrEqualTo(10)); }
Expected: assignable to <System.Int32> or assignable to <System.Double> But was: "5.0" Expected: greater than or equal to 0 and less than or equal to 10 But was: -5.0d
/// <summary> /// Syntax for executing multiple assertions in the same test (ie: all asserts are run) /// </summary> [Test] public void MultipleCriteriaChecks() { var aNumber = 5.0; aNumber.Should().BeGreaterOrEqualTo(0).And.BeLessOrEqualTo(10); aNumber.Should().BeOfType(typeof(double)).And.BeInRange(0.0, 10.0); }
Expected aNumber to be less or equal to 10.0, but found 15.0.
/// <summary> /// Syntax for executing multiple assertions in the same test (ie: all asserts are run) /// </summary> [Test] public void MultipleCriteriaChecks() { double aNumber = 5.0; aNumber.ShouldSatisfyAllConditions( () => aNumber.ShouldBeAssignableTo<double>(), () => aNumber.ShouldBeGreaterThanOrEqualTo(0.0), () => aNumber.ShouldBeLessThanOrEqualTo(10.0) ); }
Shouldly.ShouldAssertException : aNumber should satisfy all the conditions specified, but does not. The following errors were found ... --------------- Error 1 --------------- aNumber should be assignable to System.Int32 but was System.Double --------------- Error 2 --------------- aNumber should be greater than or equal to 0d but was -5.1d -----------------------------------------
Multiple Asserts
Normally, once an Assert fails, execution of the test halts and no additional Asserts are attempted. NUnit provides Assert.Multiple
, and Fluent Assertions provides AssertionScope
. You can use each to wraps your Assert calls and it will attempt to run all your Assert statements and provide a consolidated failure message with all of the Asserts that failed.
/// <summary> /// Syntax for executing multiple assertions in the same test (ie: all asserts are run) /// </summary> [Test] public void MultipleCriteriaChecks() { object aNumber = 5.0; Assert.Multiple(() => { Assert.That(aNumber, Is.AssignableTo<double>()); Assert.That(aNumber, Is.InRange(0.0, 10.0)); } ); }
Expected: assignable to <System.String> But was: <System.Double> ... Expected: in range (0,10) But was: -5.0d
/// <summary> /// Syntax for executing multiple assertions in the same test (ie: all asserts are run) /// </summary> [Test] public void MultipleCriteriaChecks() { var aNumber = 5.0; using (new AssertionScope()) { aNumber.Should().BeOfType(typeof(double)); aNumber.Should().BeInRange(0.0, 10.0); } }
Expected type to be System.String, but found System.Double. Expected aNumber to be between 0.0 and 10.0, but found 15.0.
NUnit Specialized Asserts
NUnit provides some additional Assert options that are useful in some specialized cases.
Filesystem Asserts
NUnit also provides a set of File- and Directory-specific asserts for dealing with paths and file-related data.
/// <summary> /// File- and Directory-specific checks /// </summary> [Test] public void FileChecks() { var realFilePath = Assembly.GetExecutingAssembly().Location; var realFileInfo = new FileInfo(realFilePath); var realDirectoryPath = Path.GetDirectoryName(realFilePath); var realDirectoryInfo = new DirectoryInfo(realDirectoryPath); var nonexistantFilePath = "E:/ fake.folder / this.is.fake"; var nonexistantFileInfo = new FileInfo(nonexistantFilePath); var nonexistantDirectoryPath = Path.GetDirectoryName(nonexistantFilePath); var nonexistantDirectoryInfo = new DirectoryInfo(nonexistantDirectoryPath); Assert.That(realFilePath, Does.Exist); Assert.That(realFileInfo, Does.Exist); Assert.That(nonexistantFilePath, Does.Not.Exist); Assert.That(nonexistantFileInfo, Does.Not.Exist); Assert.That(realDirectoryPath, Does.Exist); Assert.That(realDirectoryInfo, Does.Exist); Assert.That(nonexistantDirectoryPath, Does.Not.Exist); Assert.That(nonexistantDirectoryInfo, Does.Not.Exist); Assert.That(realDirectoryInfo, Is.Not.Empty); Assert.That("/folder1/./junk/../folder2", Is.SamePath("/folder1/folder2")); Assert.That("/folder1/./junk/../folder2/..", Is.Not.SamePath("/folder1/folder2")); Assert.That("/folder1/./junk/../folder2", Is.SamePath("/FOLDER1/folder2").IgnoreCase); Assert.That("/folder1/./junk/../folder2", Is.Not.SamePath("/FOLDER1/folder2").RespectCase); Assert.That("/folder1/./junk/../folder2/./foo", Is.SamePathOrUnder("/folder1/folder2")); Assert.That("/folder1/./junk/../folder2/./foo", Is.SubPathOf("/folder1")); }
Expected: file or directory exists But was: "E:/ fake.folder / this.is.fake" Expected: not file or directory exists But was: "C:\Users\wrigh\AppData\Local\NCrunch\15220\6\NUnit.FullFramework\bin\Debug\NUnit.FullFramework.dll" Expected: not file or directory exists But was: "C:\Users\wrigh\AppData\Local\NCrunch\15220\6\NUnit.FullFramework\bin\Debug" Expected: file or directory exists But was: "E:\fake.folder" Expected: not Path matching "/folder1/folder2" But was: "/folder1/./junk/../folder2" Expected: Path under or matching "/folder1/folder2/x" But was: "/folder1/./junk/../folder2/./foo"
/// <summary> /// File- and Directory-specific checks /// </summary> [Test] public void FileChecks() { var realFilePath = Assembly.GetExecutingAssembly().Location; var realFileInfo = new FileInfo(realFilePath); var realDirectoryPath = Path.GetDirectoryName(realFilePath); var realDirectoryInfo = new DirectoryInfo(realDirectoryPath); var nonexistantFilePath = "E:/ fake.folder / this.is.fake"; var nonexistantFileInfo = new FileInfo(nonexistantFilePath); var nonexistantDirectoryPath = Path.GetDirectoryName(nonexistantFilePath); var nonexistantDirectoryInfo = new DirectoryInfo(nonexistantDirectoryPath); // see: https://github.com/nunit/docs/wiki/File-Assert FileAssert.Exists(realFileInfo); FileAssert.Exists(realFilePath); FileAssert.DoesNotExist(nonexistantFileInfo); FileAssert.DoesNotExist(nonexistantFilePath); DirectoryAssert.Exists(realDirectoryPath); DirectoryAssert.Exists(realDirectoryInfo); DirectoryAssert.DoesNotExist(nonexistantDirectoryPath); DirectoryAssert.DoesNotExist(nonexistantDirectoryInfo); }
Expected: file exists But was: <E:/fake.folder/this.is.fake> Expected: not file exists But was: <C:\Users\wrigh\AppData\Local\NCrunch\15220\3\NUnit.FullFramework\bin\Debug\NUnit.FullFramework.dll> Expected: not directory exists But was: "C:\Users\wrigh\AppData\Local\NCrunch\15220\3\NUnit.FullFramework\bin\Debug" Expected: directory exists But was: "E:\fake.folder"
Fluent Assertions Extras
Fluent Assertions provides a whole lot more than what I've covered here, including assertions for enums, nullable types, dictionaries, guids, plus an extensibility API that allows you to easily introduction your own custom assertions.
All of these are documented well at https://fluentassertions.com/introduction