BT

Your opinion matters! Please fill in the InfoQ Survey!

Using vfsStream to Test File Uploads with Laravel

| Posted by Terry Rowland Follow 0 Followers on Nov 16, 2016. Estimated reading time: 14 minutes |

A note to our readers: As per your request we have developed a set of features that allow you to reduce the noise, while not losing sight of anything that is important. Get email and web notifications by choosing the topics you are interested in.

Key takeaways

  • Testing is important in application development, and test driven development is a great item to have in a developers tool belt.
  • Testing file uploads are not as easy as one may think.
  • There are many great, less well-known testing tools available today.
  • Larval makes it easy to validate requests.
  • Testing using the actual file system isn’t desired since it actually could add/remove files in the project, creating noise.

Testing of uploading files can be tricky, but with the right tools and the knowledge of a few tricks, the process can be more efficient and a lot less difficult.

If you haven’t been exposed to vfsStream in the past, it allows interaction with a file that is stored in memory vs. one that is physically on the machine. This is nice since you don't have to delete files that are used for testing, which is more of a problem when a test fails and the teardown or other code doesn't run to remove the files used in testing. Additionally, it's faster since it is dealing with memory rather than the physical hard drive. Simply put, it's cleaner and faster.

This post will go over creating an endpoint (route) to upload a CSV file of users and testing that users in the CSV are displayed in the JSON response, as well as adding validation to ensure CSV files are the types of files being dealt with. While it is an overly realistic example, it should serve as a good launching point for implementing something similar in your projects.

For this overview, it is assumed you have Laravel (we use 5.3) installed. You won't need to actually hit this route, but it is recommend you use Laravel Valet or Homestead. This guide isn't going to go over getting those up and running. Here is a link to the repository where you can grab the code for the post.

Getting Started

Opening a console and navigating to the project root folder (where you can see the composer.json file listed alongside the .env file and others), if you run

composer require mikey179/vfsStream --dev

you will pull in vfsStream into the development section of the composer file and also pull in the needed files to leverage the package in testing. And with that, you should be ready to get testing.

Let’s Get to Some Code, Using TDD

To make this easy, simply make a file under the tests folder (from the project root again), and name it UploadCsvTest.php. Now, you need to open the file and put this inside it.

<?php

class UploadCsvTest extends TestCase
{
    public function testSuccessfulUploadingCsv()
    {
        
    }

}

Start by getting the file upload working first, and then you can come back around and add the validation. The reason for doing it this way is so you can iteratively add functionality to working code. For example, if you can upload the file, that's working code, adding validation to it is enhancing it so that can be done later.

Now add the following code to "testSuccessfullUploadingCsv" function:

$this->json('POST', 'upload-new-users-csv', ['usersCsvFile' => 'someFile']);
$this->assertResponseOk();

From the console if you run this (later this will be referred to as PHPUnit)

phpunit --filter=UploadCsvTest

you should get, and expect, a failure - and it should look like this:

Expected status code 200, got 404.
Failed asserting that false is true.

Since a 404 means the route can't be found, let’s define that. Navigate to the routes/web.php file (The routes file moved here from the Http folder in previous version of Laravel) and add this to the bottom of the file:

Route::post('/upload-new-users-csv', function (\Illuminate\Http\Request $request) {
    
});

Now if you run PHPUnit, you will get a passing test. So, now it's time to access the file you should expect to get in the closure in the route file. So back in that file add the following line inside the closure:

$request->input('usersCsvFile')->openFile();

The Route should now look like this:

Route::post('/upload-new-users-csv', function (\Illuminate\Http\Request $request) {
    $request->input('usersCsvFile')->openFile();
});

Now when you run PHPUnit, you are going to get another error, a 500. Figuring out what happened here is a little tricky. You can look at the content of the response by doing

dd($this->response->getContent());

in the test, but it’s important this is done before the asserting the response is OK, otherwise the test will fail before you get the output of the dd. The other way is to look at the log file located at "storage/logs/laravel.log". Either way you check, you should see a message similar to "Call to a member function openFile() on string", which is what you want because you sent a string through when you made the post in the test. Now it's time to get up to speed on vfs!

Getting Familiar with vfsStream

By no means is this comprehensive, but it's a start, and this is a very deep and powerful package. To make the test look a little nicer, begin by making the creation of the file in a function. So go back to the test file (UploadCsvTest.php) and make it look like this:

<?php 

use Illuminate\Http\UploadedFile;
use org\bovigo\vfs\vfsStream;

class UploadCsvTest extends TestCase
{
    public function testSuccessfulUploadingCsv()
    {
    	$this->json('POST', 'upload-new-users-csv', ['usersCsvFile' => $this->createUploadFile()]);
    	$this->assertResponseOk();
    }

	protected function createUploadFile()
	{
		$vfs = vfsStream::setup(sys_get_temp_dir(), null, ['testFile' => 'someString']);

	    return new UploadedFile(
	        $vfs->url().'/testFile',
	        null,
	        'text/csv',
	        null,
	        null,
	        true
	    );
	}
}

A lot has been introduced here; first the vfsStream::setup. This is creating a file in memory. The example uses the "sys_get_temp_dir()" just to be "more realistic", but anything could be used here, an example would be changing it to the string "root", and it will work just the same. The idea behind this is to allow you to create any directory structure to suit your needs.

Here it's not really important because PHP will automatically place uploaded files in the system temp directory; hence just using the "sys_get_temp_dir()", but it really doesn't matter in this case. The next parameter is setting the file permissions. In this case you don't need to get fancy with it so simply set it to null, allowing you to do what you want with the file. Finally there's the contents of directory ("/tmp" in the example), which is nothing more than an array of file name (key) and the value (a generic string for now, but it will need to be a CSV).

Lastly, you see the UploadedFile, which is an extension of Symfony's UploadedFile, but as a quick break down of the parameters in order (from the Symfony comment block):

First parameter: The full temporary path to the file
Second parameter: The original file name
Third parameter: The type of the file as provided by PHP; null defaults to application/octet-stream
Fourth parameter: The file size
Fifth parameter: The error constant of the upload (one of PHP's UPLOAD_ERR_XXX constants); null defaults to UPLOAD_ERR_OK
Sixth parameter: Whether the test mode is active

If you run PHPUnit, you should get a passing test! Now, time to make the file look a little more realistic. In the createUploadFile function, make it look like this:

protected function createUploadFile()
{
    $vfs = vfsStream::setup(sys_get_temp_dir(), null, ['testFile.csv' => '']);

    $uploadFile = new UploadedFile(
        $vfs->url().'/testFile.csv',
        null,
        'text/csv',
        null,
        null,
        true
    );

    collect([
        ['username', 'first name', 'last name'],
        ['jondoe', 'Jon', 'Doe'],
        ['janedoe', 'Jane', 'Doe']
    ])->each(function ($fields) use ($uploadFile) {
        $uploadFile->openFile('a+')->fputcsv($fields);
    });

    return $uploadFile;
}

This leverages Laravel's collection helper to populate the file using some PHP functionality. The openFile() will get the underlying SPLFileObject and work with it directly, adding arrays to the file as if it were a line in a CSV. After running PHPUnit, you should still get green. Now, make sure you are getting what is expected, the JSON listing of the file in the response. To accomplish that, make the testSuccessfulUpload function look like this:

public function testSuccessfulUploadingCsv()
{
    $this->json('POST', 'upload-new-users-csv', ['usersCsvFile' => $this->createUploadFile()])
        ->seeJson([
            "username,\"first name\",\"last name\"\n","jondoe,Jon,Doe\n","janedoe,Jane,Doe\n"
        ]);
    $this->assertResponseOk();
}

After running PHPunit, you will get red, as expected, so make that pass. Opening the web.php file, change the route closure to be this:

Route::post('/upload-new-users-csv', function (\Illuminate\Http\Request $request) {
    $uploadedFile = $request->input('usersCsvFile')->openFile();

    $returnArray = collect();
    while (!$uploadedFile->eof()) {
        $returnArray->push($uploadedFile->fgets());
    }

    return $returnArray;
});

All this does is read the CSV from the input (post) and places it into a collection so you can spit it out. Not a good real world example, but it's enough to get the test passing. So after running PHPUnit, you should get green.

Now, it’s time to create the test that going to make things break. There's a lot of work and preparation to this than one might think.

First, start with making the "createUploadFile" function more reusable so it can continue to pass the tests. The function should now look like this:

protected function createUploadFile(vfsStreamFile $file)
    {
        return new UploadedFile(
            $file->url(),
            null,
            mime_content_type($file->url()),
            null,
            null,
            true
        );
    }

Make sure to import (use) "org\bovigo\vfs\vfsStreamFile", or you will get error when you get to a state where green is expected. This will allow you to be able to make a file using vfsSream and it will know where (in memory) it's located and also determine the MIME type to make it.

Now you can create something that will make virtual files:

protected function createVirtualFile($filename, $extension)
    {
        return vfsStream::setup(sys_get_temp_dir(), null, [$filename.'.'.$extension => '']);
    }

This is a simple function, not a lot going on. It simply makes a virtual file with the given name and extension. This helps to reduce duplication as you want to create a CSV and some other type later for testing.

Now, create a function that will utilize these functions to create a CSV, what you need now for the current test.

protected function createCsvUploadFile($fileName = 'testFile')
{
      $virtualFile = $this->createVirtualFile($fileName, 'csv')->getChild($fileName.'.csv');

      $fileResource = fopen($virtualFile->url(), 'a+');
      collect([
          ['username', 'first name', 'last name'],
          ['jondoe', 'Jon', 'Doe'],
          ['janedoe', 'Jane', 'Doe']
      ])->each(function ($fields) use ($fileResource) {
          fputcsv($fileResource, $fields);
      });
      fclose($fileResource);

      return $this->createUploadFile($virtualFile);
}

This function is a little busier than the other two previously introduced. First, you’re making a CSV file of a given name, but then you utilize the "getChild" function, which is new. What this allows you to do is "pluck" the virtual file from the virtual system so you can work with it in a more direct manner. Previously, you didn't need to do this as you fed the file path directly to the UploadFile with this:

new UploadedFile(
        $vfs->url().'/testFile.'.$extension, // This is the "manual" way of getting the child like we did above.
        null,
        text/csv', // This being hard carded in the old code makes us not be able to test different MIME types.
        null,
        null,
        true
    );

Now, back to the "createCsvUploadFile" function.  After creating the file system and getting the CSV file out of it, you can use plain old PHP and Laravel to load the file up with your old data, making a CSV. Then pass it on over to the "createUploadFile" so you can get an upload file to work within your test ("testSuccessfulUploadingCsv") function.

Change what you need here to utilize that new "createCsvUploadFile" function, which is a pretty simple change:

// This line 
$this->createUploadFile('.csv')

// Becomes 
$this->createCsvUploadFile() // you could fill the file name in if you like. I added that parameter for future use if needed or wanted.

That’s it!

So now your test file should look like this:

<?php

use Illuminate\Http\UploadedFile;
use org\bovigo\vfs\vfsStream;
use org\bovigo\vfs\vfsStreamFile;

class UploadCsvTest extends TestCase
{
    public function testSuccessfulUploadingCsv()
    {
        $this->json('POST', 'upload-new-users-csv', [
            'usersCsvFile' => $this->createCsvUploadFile()
        ]);
        $this->assertResponseOk();
        $this->seeJson(["username,\"first name\",\"last name\"\n","jondoe,Jon,Doe\n","janedoe,Jane,Doe\n"]);
    }

    protected function createCsvUploadFile($fileName = 'testFile')
    {
        $virtualFile = $this->createVirtualFile($fileName, 'csv')->getChild($fileName.'.csv');

        $fileResource = fopen($virtualFile->url(), 'a+');
        collect([
            ['username', 'first name', 'last name'],
            ['jondoe', 'Jon', 'Doe'],
            ['janedoe', 'Jane', 'Doe']
        ])->each(function ($fields) use ($fileResource) {
            fputcsv($fileResource, $fields);
        });
        fclose($fileResource);

        return $this->createUploadFile($virtualFile);
    }

    protected function createUploadFile(vfsStreamFile $file)
    {
        return new UploadedFile(
            $file->url(),
            null,
            mime_content_type($file->url()),
            null,
            null,
            true
        );
    }

    protected function createVirtualFile($filename, $extension)
    {
        return vfsStream::setup(sys_get_temp_dir(), null, [$filename.'.'.$extension => '']);
    }
}

If you run the tests, you should get green (as expected).

Let’s get Back to Doing Some TDD

Now you can focus on trying to break the upload with a non CSV file. Since PHP can make image files relatively easily,that would be a good choice. Make a function that can make that kind of file and feed it into an upload file:

protected function createImageUploadFile($fileName = 'testFile', $extension = 'jpeg')
    {
        $virtualFile = $this->createVirtualFile($fileName, $extension)->getChild($fileName.'.'.$extension);
        imagejpeg(imagecreate(500, 90), $virtualFile->url());
        return $this->createUploadFile($virtualFile);
    }

A pretty straight forward function here. Normally you may have just hardcoded the file type into the creation of the file, and in the getChild, but you want to try to trick this upload. This is done to test if there is an image - with a csv extension - to make sure it doesn't get fooled. So, after making the empty file, you will notice a jpeg (via the imagejpeg and imagecreate functions) in the file and then hand it over to the creation of the "createUploadFile" function, and back to its caller it goes. So, now all that's left to get some errors going is making the new function:

public function testOnlyCsvsCanBeUploadedRegardlessOfExtension()
{
    $this->json('POST', 'upload-new-users-csv', [
        'usersCsvFile' => $this->createImageUploadFile('testFile', 'csv')
     ]);
     $this->assertResponseStatus(422);
}

This example is trying to upload that tricky file, a image file with a csv extension and you would expect a 422 (Unprocessable Entity). If you run phpunit, you should get an error, something like "Expected status code 422, got 500." This is perfect; it failed exactly as expected.

Now it's time to leverage Laravel's Form Requests. You can use Artisan to do this easily. Back in the console, if you run:

php artisan make:request UploadCsvUsers

Laravel will make a file under app/Http/Requests named UploadCsvUsers.php. Opening that file up, you will see:

<?php

namespace App\Http\Requests;

use Illuminate\Foundation\Http\FormRequest;

class UploadCsvUsers extends FormRequest
{
    /**
     * Determine if the user is authorized to make this request.
     *
     * @return bool
     */
    public function authorize()
    {
        return false;
    }

    /**
     * Get the validation rules that apply to the request.
     *
     * @return array
     */
    public function rules()
    {
        return [
            //
        ];
    }
}

If you would like, read up more on these here. Change the "authorize" function to return true by default (if the function returns false it will stop the request and not run the rules). For now, just leave the rules empty and jump over to your routes/web.php file and add the form request to the route to see what happens. Your web.php file should now look similar to this:

use App\Http\Requests\UploadCsvUsers;

Route::post('/upload-new-users-csv', function (UploadCsvUsers $request) {
    $uploadedFile = $request->usersCsvFile->openFile();


    $returnArray = collect();
    while (!$uploadedFile->eof()) {
        $returnArray->push($uploadedFile->fgets());
    }

    return $returnArray;
});

Giving the code a run in phpunit, nothing's changed, you still fail. So, to avoid running off the rails here, you can check out all the available options for Laravel's validation here, but for now focus on the 'MIME Type By File Extension' rules. So, edit your Request and make it look like this:

public function rules()
{
      return [
     		'usersCsvFile' => 'required|mimes:csv,txt'
      ];
}

Since you know the filename you want to use, use that as the key and add a couple rules here. The first one says the file is required for the upload, which is not being tested here but feel free to make a test of your own. Simply make a test that doesn't have the "usersCsvFile" in the upload parameters and you will see the request in action. The mimes rule here will accept a csv or text, since csv files are simply text.

Now if you run PHPUnit, you will get all green and you are officially done.

This is a pretty good test. Not only will this read CSVs like you want, but it also won't accept a file disguised as a CSV file.

Conclusion

When first uploading file tests and MIME type validation, it is easy to get thrown for a loop. This is especially true when one is used to using acceptance (browser) tests to do this, however the execution time of the test and effort needed to get those kind of tests built to do something that had very little (or nothing) to do with the actual user interface of the application was too high.

This is a way to do the testing faster and lighter. While this is by no means a fully explored example, it's a good place to start. Additionally, if you needed to do image uploads, with a little work, you could flip some things around and things should work the same.

About the Author

Terry Rowland is a Senior Web Application Developer at Enola Labs, an Austin, Texas-based web and mobile app development company.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT