Work with files and directories in a .NET app

Introduction

When you are working on a .NET application, you may need to work with files and directories. In this post, we will Build an app that manipulates files and directories with C# and .NET.

• We will:
  • Create and work with directories.
  • Create and delete files.
  • Read from files.
  • Write to files.
  • Parse data in files.

Project Overview

• Let’s assume that you work for a company called Tailwind Traders, the second-largest online retailer in the world
• Their IT operations department deals with mountains of data and files.
• The company has hired you to help it manage its data and files by using C# and .NET.
• We’ll write a program that searches through folders for sales files. When those files are found, we’ll use C# and .NET to:
  • read and parse the sales-total data in those files.
  • Finally, we’ll summarize the sales totals into one grand total and
  • write that value to a new file.

System.IO Namespace

• .NET contains built-in types for working with the file system that you can find in the System.IO namespace.
• Large retailers often write data to files so it can be processed later in batches.
• Tailwind Traders has each of its stores write its sales total to a file and send that file to a central location.
• To use those files, the company needs to create a batch process that can work with the file system.
• The System.IO namespace contains built-in types that allow you to interact with files and directories.
  • For example, you can retrieve collections of files and directories based on search criteria and get and set properties for files and directories.
  • You can also use System.IO namespace types to synchronously and asynchronously read and write data streams and files.
• The Directory class exposes static methods for creating, moving, and enumerating through directories and subdirectories.

List all directories

• The Directory class is often used to list out (or enumerate) directories.
  • For instance, the Tailwind Traders file structure has a root folder called stores.
  • In that folder are subfolders organized by store number, and
  • inside those folders are the sales-total and inventory files.
  • The structure looks like this example:

📂 stores
    📄 sales.json
    📄 totals.txt
    📂 201
       📄 sales.json
       📄 salestotals.json
       📄 inventory.txt
    📂 202

• To read through and list the names of the top-level directories, use the Directory.EnumerateDirectories function.

IEnumerable<string> listOfDirectories = Directory.EnumerateDirectories("stores");

foreach (var dir in listOfDirectories) {
    Console.WriteLine(dir);
}

// Outputs:
// stores/201
// stores/202

List files in a specific directory

• To list the names of all of the files in a directory, you can use the Directory.EnumerateFiles function.

IEnumerable<string> files = Directory.EnumerateFiles("stores");

foreach (var file in files)
{
    Console.WriteLine(file);
}

// Outputs:
// stores/totals.txt
// stores/sales.json

List all content in a directory and all subdirectories

• Both the Directory.EnumerateDirectories and Directory.EnumerateFiles functions have an overload that accepts a parameter to specify that search pattern files and directories must match.
• They also have another overload that accepts a parameter to indicate whether to recursively traverse a specified folder and all of its subfolders.

// Find all *.txt files in the stores folder and its subfolders
IEnumerable<string> allFilesInAllFolders = Directory.EnumerateFiles("stores", "*.txt", SearchOption.AllDirectories);

foreach (var file in allFilesInAllFolders)
{
    Console.WriteLine(file);
}

// Outputs:
// stores/totals.txt
// stores/201/inventory.txt

Example 1 - A function to find the sales.json files in the stores folder and its subfolders

using System.IO;
using System.Collections.Generic;

var salesFiles = FindFiles("stores");

foreach (var file in salesFiles)
{
    Console.WriteLine(file);
}

IEnumerable<string> FindFiles(string folderName)
{
    List<string> salesFiles = new List<string>();

    var foundFiles = Directory.EnumerateFiles(folderName, "*", SearchOption.AllDirectories);

    foreach (var file in foundFiles)
    {
        // The file name will contain the full path, so only check the end of it
        if (file.EndsWith("sales.json"))
        {
            salesFiles.Add(file);
        }
    }

    return salesFiles;
}

Work with file paths in .NET

• .NET has a built-in mechanism for working with file-system paths.
• If you have a file system with many files and folders, manually building paths can be tedious.

Determine the current directory

• .NET exposes the full path to the current directory via the Directory.GetCurrentDirectory method.

string currentDirectory = Directory.GetCurrentDirectory();
Console.WriteLine(currentDirectory);

// Outputs:
// C:\Users\username\source\repos\TailwindTraders

Work with special directories

• .NET runs everywhere: on Windows, macOS, Linux, and even on mobile operating systems like iOS and Android.
• Each operating system might or might not have the concept of special system folders (such as a home directory—which is dedicated for user-specific files—or a desktop directory, or a directory for storing temporary files).
• Those types of special directories differ for each operating system.
• It would be cumbersome to try to remember each operating system’s directory structure and perform switches based on the current OS.
• The System.Environment.SpecialFolder enumeration specifies constants to retrieve paths to special system folders.
• The following code returns the path to the equivalent of the Windows My Documents folder, or the user’s HOME directory for any operating system, even if the code is running on Linux:

string myDocuments = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);

Work with paths

• Paths are a subject that comes up so frequently that .NET includes a class called Path specifically for working with them.
• The Path class is located in the System.IO namespace of .NET, and doesn’t need to be installed.

Special path characters

• Different operating systems use different characters to separate directory levels.
• For example,
  • Windows uses the backslash (stores\201) and
  • macOS uses the forward slash (stores/201).
• To help you use the correct character, the Path class contains the DirectorySeparatorChar field.
• .NET automatically interprets that field into the separator character that’s applicable to the operating system when you need to build a path manually.

Console.WriteLine($"stores{Path.DirectorySeparatorChar}201");

// returns:
// stores\201 on Windows
//
// stores/201 on macOS

Join paths

• The Path class works with the concept of file and folder paths, which are just strings.
• You can use the Path class to automatically build correct paths for specific operating systems.
• For instance, if you want to get the path to the stores/201 folder, you can use the Path.Combine function to do that.

string path = Path.Combine("stores", "201");
Console.WriteLine(path);

• you should use the Path.Combine or Path.DirectorySeparatorChar class instead of hard-coding strings, because your program might be running on many different operating systems.
• The Path class always formats the paths correctly for the operating system on which it’s running.
• The Path class doesn’t care whether things actually exist. Paths are conceptual, not physical, and the class is building and parsing strings for you.

Determine filename extensions

• The Path class can also tell you a filename’s extension. If you have a file and you want to know if it’s a JSON file, you can use the Path.GetExtension function.

Console.WriteLine(Path.GetExtension("sales.json")); // outputs: .json

Get everything you need to know about a file or path

• The Path class contains many different methods that do various things.
• You can get the most information about a directory or a file by using the DirectoryInfo or FileInfo classes, respectively.

string fileName = $"stores{Path.DirectorySeparatorChar}201{Path.DirectorySeparatorChar}sales{Path.DirectorySeparatorChar}sales.json";

FileInfo info = new FileInfo(fileName);

Console.WriteLine($"Full Name: {info.FullName}{Environment.NewLine}Directory: {info.Directory}{Environment.NewLine}Extension: {info.Extension}{Environment.NewLine}Create Date: {info.CreationTime}"); // And many more properties

Example 2 - Use the current directory and combine paths

• The .NET Path class and Directory.GetCurrentDirectory are two ways to define and compose file-system paths.
• In this example, we will improvee the previous example by using the Path class and Directory.GetCurrentDirectory to define and compose file-system paths.
• In the current Program.cs code (in example 1 above), we’re passing the static location of the stores folder.
• Now, we’ll change that code to use the Directory.GetCurrentDirectory value instead of passing a static folder name.
• Insert the following code above the first line of Program.cs file. This code will get the current directory and store it in a variable called currentDirectory.

var currentDirectory = Directory.GetCurrentDirectory();

• Insert the following code after the line that you just added. This code uses the Path.Combine method to create the full path to the stores directory and store it in a new variable storesDirectory:

var storesDirectory = Path.Combine(currentDirectory, "stores");

• Replace the string stores in the FindFiles function call with the new variable storesDirectory:

var salesFiles = FindFiles(storesDirectory);

• All the code together should look like this:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");
var salesFiles = FindFiles(storesDirectory);

foreach (var file in salesFiles)
{
    Console.WriteLine(file);
}

IEnumerable<string> FindFiles(string folderName)
{
    List<string> salesFiles = new List<string>();

    var foundFiles = Directory.EnumerateFiles(folderName, "*", SearchOption.AllDirectories);

    foreach (var file in foundFiles)
    {
        // The file name will contain the full path, so only check the end of it
        if (file.EndsWith("sales.json"))
        {
            salesFiles.Add(file);
        }
    }

    return salesFiles;
}

• Now, when you run the program, it will use the current directory to find the stores directory and then search for sales.json files in that directory and its subdirectories.
• What if we want to find ALL .json files in the stores directory and its subdirectories?

Example 3 - Find all .json files in the stores directory and its subdirectories

• Instead of looking for only sales.json files, the program needs to search for any file with a .json extension.
• To do that, you can use the Path.GetExtension method to check the extension for each file.
• In the foreach loop that iterates through foundFiles, insert the following line of code above the if statement to define a new variable extension.
• This code uses the Path.GetExtension method to get the extension of each file.

var extension = Path.GetExtension(file);

• Change the if statement to look like the following line of code. This statement checks whether the file’s extension is equal to .json.

if (extension == ".json")

• The foreach loop should look similar to the following code:

foreach (var file in foundFiles)
{
    var extension = Path.GetExtension(file);

    if (extension == ".json")
    {
        salesFiles.Add(file);
    }
}

• The full code should look like this:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesFiles = FindFiles(storesDirectory);

foreach (var file in salesFiles)
{
    Console.WriteLine(file);
}

IEnumerable<string> FindFiles(string folderName)
{
    List<string> salesFiles = new List<string>();

    var foundFiles = Directory.EnumerateFiles(folderName, "*", SearchOption.AllDirectories);

    foreach (var file in foundFiles)
    {
        var extension = Path.GetExtension(file);
        if (extension == ".json")
        {
            salesFiles.Add(file);
        }
    }

    return salesFiles;
}

• Now, when you run the program, it will search for all .json files in the stores directory and its subdirectories.

Create files and directories

• Creating and deleting new files and directories programmatically is a common requirement for line-of-business applications.
• You can also use the Directory class to create, delete, copy, move, and otherwise manipulate directories on a system programmatically.
• You can use an analogous class called File to do the same on files.

Create a directory

• Use the Directory.CreateDirectory method to create directories.
• The following method creates a new folder called newDir inside the 201 folder:

Directory.CreateDirectory(Path.Combine(Directory.GetCurrentDirectory(), "stores","201","newDir"));

• If /stores/201 doesn’t already exist, it’s created automatically.
• The CreateDirectory method doesn’t fail. It creates any directories and subdirectories passed to it.

Make sure directories exist

• Sometimes, you need to check if a directory already exists.
• For example, you might need to check before you create a file in a specified directory to avoid an exception that could cause your program to stop abruptly.
• To see if a directory exists, use the Directory.Exists method:

bool doesDirectoryExist = Directory.Exists(filePath);

Create a file

• You can create files by using the File.WriteAllText method.
• This method takes in a path to the file and the data you want to write to the file.
• If the file already exists, it’s overwritten.
• For instance, this code creates a file called greeting.txt with the text “Hello World!” inside:

File.WriteAllText(Path.Combine(Directory.GetCurrentDirectory(), "greeting.txt"), "Hello World!");

Example 4 - Create a new directory and file

• In this example, we will further enhance the previous example by creating the salesTotalDir directory and the totals.txt file where the sales totals are collated.
• In the Program.cs file, remove the foreach loop that iterates and writes each filename returned from the FindFiles function to the Console output.
• In the Program.cs file, create a variable called salesTotalDir, which holds the path to the salesTotalDir directory:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesTotalDir = Path.Combine(currentDirectory, "salesTotalDir");

var salesFiles = FindFiles(storesDirectory);

• In the Program.cs file, add code to create the directory:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesTotalDir = Path.Combine(currentDirectory, "salesTotalDir");
Directory.CreateDirectory(salesTotalDir);   // Add this line of code

var salesFiles = FindFiles(storesDirectory);

Write the totals.txt file

• In the Program.cs file, add the code to create an empty file called totals.txt inside the newly created salesTotalDir directory.
• Use an empty string for the file’s contents for now:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesTotalDir = Path.Combine(currentDirectory, "salesTotalDir");
Directory.CreateDirectory(salesTotalDir);

var salesFiles = FindFiles(storesDirectory);

File.WriteAllText(Path.Combine(salesTotalDir, "totals.txt"), "");   // Add this line of code

• The full code should look like this:

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesTotalDir = Path.Combine(currentDirectory, "salesTotalDir");
Directory.CreateDirectory(salesTotalDir);
var salesFiles = FindFiles(storesDirectory);

File.WriteAllText(Path.Combine(salesTotalDir, "totals.txt"), String.Empty);

IEnumerable<string> FindFiles(string folderName)
{
    List<string> salesFiles = new List<string>();

    var foundFiles = Directory.EnumerateFiles(folderName, "*", SearchOption.AllDirectories);

    foreach (var file in foundFiles)
    {
        var extension = Path.GetExtension(file);
        if (extension == ".json")
        {
            salesFiles.Add(file);
        }
    }

    return salesFiles;
}

Read and write to files

• Reading data from files and writing data to files are core concepts in .NET.

Read from a file

• Files are read through the ReadAllText method on the File class.

File.ReadAllText($"stores{Path.DirectorySeparatorChar}201{Path.DirectorySeparatorChar}sales.json");

• The ReadAllText method reads the contents of the file and returns it as a string.

{
  "total": 22385.32
}

Parse data in files

• This data in its string format doesn’t do you much good. It’s still just characters, but now in a format that you can read. You want the ability to parse this data into a format that you can use programmatically.
• There are many ways to parse JSON files with .NET, including a community library known as Json.NET.
• You can add the Json.NET package to your project by using NuGet:

dotnet add package Newtonsoft.Json

• Then, add using Newtonsoft.Json to the top of your class file:

using Newtonsoft.Json;

• And use the JsonConvert.DeserializeObject method:

var salesJson = File.ReadAllText($"stores{Path.DirectorySeparatorChar}201{Path.DirectorySeparatorChar}sales.json");
var salesData = JsonConvert.DeserializeObject<SalesTotal>(salesJson);

Console.WriteLine(salesData.Total);

class SalesTotal
{
  public double Total { get; set; }
}

• Files come in a variety of formats. JSON files are the most desirable to work with because of the built-in support in the language.
• You also might encounter files that are .csv, fixed width, or some other format.
• In that case, it’s best to search nuget.org for a parser for that file type.

Write to a file

• To write data to a file, use the same WriteAllText method, but pass in the data that you want to write.

var data = JsonConvert.DeserializeObject<SalesTotal>(salesJson);

File.WriteAllText($"salesTotalDir{Path.DirectorySeparatorChar}totals.txt", data.Total.ToString());

// totals.txt
// 22385.32

Append to a file

• In the preceding example, the file is overwritten every time you write to it.
• Sometimes, you don’t want that. You want to append data to the file instead of replacing it entirely.
• You can append data with the File.AppendAllText method. By default, File.AppendAllText creates the file if it doesn’t already exist.

var data = JsonConvert.DeserializeObject<SalesTotal>(salesJson);

File.AppendAllText($"salesTotalDir{Path.DirectorySeparatorChar}totals.txt", $"{data.Total}{Environment.NewLine}");

// totals.txt
// 22385.32
// 22385.32

• Environment.NewLine prompts .NET to put the value on its own line. If you didn’t pass this value, you would get all the numbers squished together on the same line.

Example 5 - Read from a file, parse the data, and write to a new file

• We will now enhance the previous example by reading the sales.json files, parsing the data, and writing the total sales to the totals.txt file.
• We will:
  • Add Json.NET to the project.
  • create a method to calculate the total sales.
  • call the method
  • write the total sales to the totals.txt file.
• The final code should look like this:

using Newtonsoft.Json;

var currentDirectory = Directory.GetCurrentDirectory();
var storesDirectory = Path.Combine(currentDirectory, "stores");

var salesTotalDir = Path.Combine(currentDirectory, "salesTotalDir");
Directory.CreateDirectory(salesTotalDir);

var salesFiles = FindFiles(storesDirectory);

var salesTotal = CalculateSalesTotal(salesFiles);

File.AppendAllText(Path.Combine(salesTotalDir, "totals.txt"), $"{salesTotal}{Environment.NewLine}");

IEnumerable<string> FindFiles(string folderName)
{
    List<string> salesFiles = new List<string>();

    var foundFiles = Directory.EnumerateFiles(folderName, "*", SearchOption.AllDirectories);

    foreach (var file in foundFiles)
    {
        var extension = Path.GetExtension(file);
        if (extension == ".json")
        {
            salesFiles.Add(file);
        }
    }

    return salesFiles;
}

double CalculateSalesTotal(IEnumerable<string> salesFiles)
{
    double salesTotal = 0;

    // Loop over each file path in salesFiles
    foreach (var file in salesFiles)
    {
        // Read the contents of the file
        string salesJson = File.ReadAllText(file);

        // Parse the contents as JSON
        SalesData? data = JsonConvert.DeserializeObject<SalesData?>(salesJson);

        // Add the amount found in the Total field to the salesTotal variable
        salesTotal += data?.Total ?? 0;
    }

    return salesTotal;
}

record SalesData (double Total);