Ubuntu uses the Debian packaging system which is designed to keep the configuration files of uninstalled applications. This feature frees you from redoing the configuration when reinstalling an application.

However, if you want to free up some disk space, you can easily remove those configuration files by using the following command:

sudo dpkg -P $(dpkg -l | awk '/^rc/{print $2}')

Basically, this command searches for all unused configuration files and then deletes them.

Moreover, it is worth noting that the aforementioned command doesn't delete the personal configuration files which are usually located in your home folder.

If you want to perform a complete removal of an existing (i.e. installed) application, run the following command in a terminal:

sudo apt-get purge APPLICATION_NAME