{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n# Test Histogram Plots with Matplotcheck\n\nBelow you will find some examples of how to use MatPlotCheck\nto test histogram plots created with Matplotlib in Python.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\nYou will start by importing the required packages and plotting a histogram.\nOnce you have created your plot, you will created a Matplotcheck\n``PlotTester`` object by providing the Matplotlib axis object to\n``PlotTester``.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\nimport matplotcheck.base as mpc\nimport numpy as np\n\n\ndata = np.exp(np.arange(0, 5, 0.01))\n\nfig, ax = plt.subplots()\nax.hist(data, bins=5, color=\"gold\")\n\n# Create a Matplotcheck PlotTester object\nplot_tester_1 = mpc.PlotTester(ax)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test a Histogram Plot\nOnce you have created a PlotTester object, you are ready to test various\nparts of your plot. Below, you test both\nthe number of bins and the values associated with those bins.\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Note

Throughout this vignette, the term `bin value` is used to describe the\n number of datapoints that fall within a bin. In other words, a bin's value\n is equal to the height of the bar corresponding to that bin. For example,\n the value of the first bin in the above histogram is 341. Note that the\n height of the first bar is also 341.

\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Test that the histogram plot has 5 bins\nplot_tester_1.assert_num_bins(5)\n\n# Test that the histogram bin values (the height of each bin) is as expected\nexpected_bin_values = [341, 68, 40, 28, 23]\nplot_tester_1.assert_bin_values(expected_bin_values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And you can also run some tests that will fail.\n\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "try:\n plot_tester_1.assert_num_bins(6)\nexcept AssertionError as message:\n print(\"AssertionError:\", message)\n\ntry:\n plot_tester_1.assert_bin_values([1, 4, 1, 3, 4])\nexcept AssertionError as message:\n print(\"AssertionError:\", message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Determining Expected Values\nYou can use the MatPlotCheck ``get_bin_values()`` method to extract the bin\nvalues that are expected for a plot. This is helpful if you are using a tool\nlike nbgrader to create the the expected plot outcomes in a homework\nassignment.\n\nTo extract bin values from an expected plot you first create the expected\nhistogram plot that you will use to grade your assignment (or htat you expect\nas an outcome from a test). Next, you create a PlotTester object from that\nplot. Finally, you call the ``get_bin_values()`` method to grab the expected\nbin values from that plot.\n\nThe steps outlined above are implemented below.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "expected_data = np.sin(np.arange(0, 2 * np.pi, np.pi / 50))\n\n# Create the expected plot\nfig, ax = plt.subplots()\nax.hist(expected_data, bins=8, color=\"gold\")\n\n# Create a Matplotcheck PlotTester object from the axis object\nplot_tester_expected = mpc.PlotTester(ax)\n# Get bin values from the expected plot\nprint(plot_tester_expected.get_bin_values())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This example assumes that you are creating tests for a student\nassignment. Once you have created the PlotTester object for the expected\nplot (this is the answer to the assignment that you expect the student to\ncome to),\nyou can then test the student plot to see if it matches expected bin values.\nBelow another plot is created that represents the student submitted plot.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Create and plot the student submitted histogram\ndata = np.sin(np.arange(2 * np.pi, 4 * np.pi, np.pi / 50))\nfig, ax = plt.subplots()\nax.hist(data, bins=8, color=\"orange\")\n\n# Test the student submitted histogram bin values against the expected\n# bin values (the correct answer to the assigned plot)\nplot_tester_testing = mpc.PlotTester(ax)\nplot_tester_testing.assert_bin_values(\n [23.0, 10.0, 8.0, 9.0, 9.0, 8.0, 10.0, 23.0]\n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Above, ``assert_bin_values()`` did not raise an ``AssertionError``. This\nmeans that the test passed and the student submitted plot has the correct\nhistogram bins.\n\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Note

In this example, you created the expected histogram (the homework answer)\n and the student submitted histogram in the same file.

\n\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing with Tolerances\nIn some cases, you might want to run a test that doesn't require the bin\nvalues to match exactly. For example, it might be ok if the values are\na few tenths off. To allow for some \"wiggle room\" in the expected answer,\nyou can use the ``tolerance`` parameter of the ``assert_bin_values()``\nmethod.\n\nYou will start by making two histograms with slightly different data and\nstoring the plots with ``nb.convert_axes()``. The gold plot will serve as the\nexpected plot, and the orange plot will serve as the testing plot.\n\nYou will then create a `PlotTester` object for each plot. This allows you to\nextract the expected bin values from the expected plot and use those value to\ntest the testing plot.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "expected_data = 0.1 * np.power(np.arange(0, 10, 0.1), 2)\nbins = np.arange(0, 10, 1)\n\nfig1, ax1 = plt.subplots()\nax1.hist(expected_data, color=\"gold\", bins=bins)\n\n# Create plot tester object\nplot_tester_expected_1 = mpc.PlotTester(ax1)\n# Get expected bin values\nbins_expected_1 = plot_tester_expected_1.get_bin_values()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test_data = 0.1995 * np.power(np.arange(0, 10, 0.1), 1.7)\nfig2, ax2 = plt.subplots()\nax2.hist(test_data, color=\"orange\", bins=bins)\n# Create plot tester object\nplot_tester_testing_2 = mpc.PlotTester(ax2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You'll notice that the test (orange) plot differs somewhat from the\nexpected (gold) plot, but still has a similar shape and similar bin\nvalues.\n\nIf you test it without the ``tolerance`` argument, the assertion will fail.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "try:\n plot_tester_testing_2.assert_bin_values(bins_expected_1)\nexcept AssertionError as message:\n print(\"AssertionError:\", message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, if you set a tolerance, the assertion can pass. Here you will test\nit with ``tolerance=6``, as that is the maximum difference between the two\ndatasets.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "plot_tester_testing_2.assert_bin_values(bins_expected_1, tolerance=6)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because no ``AssertionError`` is raised, you know that the test passed with\na tolerance of 0.2. However, the test will not pass with a tolerance that is\ntoo small; the test will fail with ``tolerance=0.1``.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "try:\n plot_tester_testing_2.assert_bin_values(bins_expected_1, tolerance=0.1)\nexcept AssertionError as message:\n print(\"AssertionError:\", message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Note

When using tolerances, the ``tolerance`` argument is taken as a relative\n tolerance. For more information, see the documentation for the\n ``base.assert_bin_heights()`` method.

\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Histogram Midpoints\nSo far, you have tested the histogram values as well as the number of bins\nthe histogram has. It may also be useful to test that the data bins cover\nthe range of values that they were expected to. In order to do this, you can\ntest the midpoints of each bin to ensure that the data covered by each\nbin is as expected. This is tested very similarly to the bins values.\nSimply provide ``assert_bin_midpoints()`` with a list of the expected\nmidpoints, and it will assert if they are accurate or not. In order to obtain\nthe midpoints in a PlotTester object, you can use ``get_bin_midpoints()``,\nmuch like ``get_bin_values()``.\n\nFor this example, you will create a plot tester object from a histogram plot,\nthe same way you did for the bin values example.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "fig, ax = plt.subplots()\nax.hist(test_data, bins=8, color=\"gold\")\n\n# If you were running this in a notebook, the commented out line below would\n# store the matplotlib object. However, in this example, you can just grab the\n# axes object directly.\n\n# midpoints_plot_hold = nb.convert_axes(plt, which_axes=\"current\")\n\nplot_tester_expected_3 = mpc.PlotTester(ax)\nprint(plot_tester_expected_3.get_bin_midpoints())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You got the values from the plot tester object! As you can see, the values\nthat were collected are the midpoints for the values each histogram bin\ncovers. Now you can test that they are asserted indeed correct with an\nassertion test.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "try:\n plot_tester_expected_3.assert_bin_midpoints(\n [-0.875, -0.625, -0.375, -0.125, 0.125, 0.375, 0.625, 0.875]\n )\nexcept AssertionError as message:\n print(\"AssertionError:\", message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here you can see that this will fail when given incorrect values.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "try:\n plot_tester_expected_3.assert_bin_midpoints(\n [-0.75, -0.5, -0.25, -0, 0.25, 0.5, 0.75, 1]\n )\nexcept AssertionError as message:\n print(\"AssertionError:\", message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Note

Keep in mind this test is for the midpoints of the range that each bin\n covers. So if a bin covers all data that's in between 0 and 1, than the\n value given for that bin will be .5, not 0 or 1.

\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# .. note::\n# If you are working on tests for jupyter notebooks, you can call the\n# line below to capture the student cell in a notebook. Then you can\n# Use that object for testing.\n# testing_plot_2_hold = nb.convert_axes(plt, which_axes=\"current\")." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" } }, "nbformat": 4, "nbformat_minor": 0 }