Publish a Haskell package in Hackage

Haskell being in the 38th position in the Programming Community Index, it has an immense number of packages providing a variety of functionalities for its developers. You can find the full list of Haskell packages in here. Even though, there are such a huge number of packages many new packages are added to Hackage frequently. 

First, let's understand what does it mean by 'package' in Haskell. A package is the unit of distribution for the Cabal, which is a system for building and packaging Haskell libraries and programs. Its purpose in life, when installed, is to make available some Haskell modules to be imported into a Haskell program a developer is writing.

Before starting off with package development, you need to understand couple of basic rules and regulations in a Haskell package. The main concern in developing a package is that, it should be light-weighted as much as possible, i.e. the build depends section in the cabal file should contain minimal dependencies to the other packages. And the other main thing, you need to be aware of is the documentation. Use 'Haddock' to build the necessary documentation of your package. Refer to this link to read more on the Haddock usage in your Haskell package.

Now that you have a basic understanding about the Haskell packages, let's get to work! 

First step is to make sure your program compiles without any errors. And then, it provides the expected output when run with the desired input parameters. 

Then, create a comprehensive cabal file for your package. You can create a cabal file for your project by running 'cabal init' in your terminal at the project directory. Once you have the basic structure, fill in the required fields of the cabal file using a text editor. After filling out fields, you can check whether your cabal file is correct by running 'cabal check' in the terminal. If there are any missing fields, this command will point out them to you.

After that, use Haddock to build the necessary documentation for the code. To do this, you can type in 'cabal haddock' in your terminal after navigating to the project directory. This command will output the document coverage of the code and points out the necessary places, which need to be updated.

If you don't have any idea how to create a comprehensive documentation for your package, just take a look at the available packages in Hackage and adapt those documentation styles to your package. 

The next step is to create the tarball of your package content. First get your code, cabal file, README file and put them in a folder with the name package-version. For an example; foo-0.1.3.0. You can read more about the package version policy in Haskell in this link. 

Since I'm using Windows, I used 7-Zip to create a tarball. Following the instructions in their you can create the tarball for you package.

Then comes the part, where you upload the package to Hackage. In order to upload a package, you must have an account. You have to request an account by filling out the form in this link. Once you have your account, log into your account.

There are 2 package upload scenarios; upload a new package OR upload a new version of an existing package.

Scenario 1: Upload a new package.

You can simply go to the upload link in your account, and click on 'Upload a package' leading you here. You can upload you tarball here, and if there are no errors in your tarball structure this will upload the package candidate smoothly. Then go your user-account. Now you can see the uploaded package. If you can't find a link to go to your account, just search your username in the user-accounts list here. 

Now go to your package and go to 'Maintainer's corner' and click on 'edit package information' which will take you to the publishing page as shown below;

Now you can publish the candidate by clicking on 'Publish candidate', making the package available for the other Haskell developers. 

Scenario 2: Upload a new version of an existing package

In order to do this you should be part of the maintainer group of the package. Given that you are in the maintainer group, you can upload a package candidate following the above steps. But the most important thing to remember is that, you cannot upload the same version number again to Hackage.

There are many resources available in the internet, you can read to get a thorough understanding on the matter. Please do read and understand the procedure before you publish your package, because you can upload a package version once!

Using Basic HTTP Authentication in Haskell

I have recently developed a Haskell library to interact with the Mailchimp JSON API version 2.0. This is the blog post regarding the implementation of the said library. At the end of the blog post I stated, that I will try and update the library to support the version 3.0 of the JSON API. Since the version 2.0 is deprecated long ago and I had some free time in hand, I decided to develop the library to work with version 3.0 API calls. To read further more about the version 3.0 refer to this link.

The major difference in version 3.0 API from the version 2.0 API you need to know in developing the library is that, the removal of API Key from the request body and introducing HTTP Basic Auth in addition to OAuth2. In this implementation, I have used Basic HTTP Auth imported from 'Network.HTTP.Conduit' module in 'http-conduit' package. For further reading on the differences between the two Mailchimp API versions, please follow this link.

The major component in developing the library for version 3.0 was to integrate Basic HTTP authentication with the API calls.  In this post, I'll take you through the implementation of Basic HTTP Auth in the library.

There are 2 basic HTTP requests we need to perform in this library. They are GET and POST requests. For GET requests, the request body is empty while for POST requests we need to send data along with the request. I have implemented 2 separate methods to perform these requests in the library. 

The POST requests in Mailchimp JSON API version 3.0 looks as follows;

    curl --request POST \

  --url 'https://dc.api.mailchimp.com/3.0/campaigns' \

  --user 'anystring:apikey' \

  --header 'content-type:application/json' \

  --data '{"recipients":{"list_id":"et43235dg"},   "type":"regular","settings":{"subject_line":"this is the subject","reply_to":"reply@gmail.com","from_name":"from Mailchimp"}}' \

  --inlcude

So there are two parts for the HTTP request, i.e. authenticate the user and send the json data to the given URL. I used the the 'applyBasicAuth' function in the 'Network.HTTP.Conduit' module. The type definition for this function is as follows;

applyBasicAuth :: ByteString -> ByteString -> Request -> Request

The parameters are passed as 'applyBasicAuth 'username' 'apiKey' $ fromJust $ parseUrl url'. Here, what 'applyBasicAuth' function does is attach a Basic Auth Header to the given Request.

In Mailchimp the username for the HTTP authentication is 'anystring', so I have hard-coded it within the functions.

Then we inject the json data to the request body of the resulting Request and the HTTP method to POST. Finally using 'httpLbs' method in 'Network.HTTP.Conduit' module we perform the action we intended. The full code snippet for this function is as follows;



For HTTP GET, the format of the request specified in Mailchimp JSON API version 3.0 is as follows;

    curl --request GET \

  --url 'https://dc.api.mailchimp.com/3.0/lists' \

  --user 'anystring:apikey'

This request retrieves data for all fields regarding each mailing list in the given account. But, if you want to receive only a couple of fields you can specify them in the request. To receive only the list ID and name for each list in the account, the request can be modified as follows;

    curl --request GET \

  --url 'https://dc.api.mailchimp.com/3.0/lists?fields=lists.id,lists.name' \

  --user 'anystring:apikey'

For the GET requests, what I have done is; remove json data injection for the initial request and setting the HTTP method as GET instead of POST. So the function to process HTTP GET request is as follows;

You can get the full program from my GitHub repository. I hope this post will be helpful for you!! Enjoy :)

Handling Dependencies in Haskell

At each Haskell compilation using a .cabal file, the compiler checks for the dependencies and downloads them. This sometimes leads to duplication of installation/ re-installation, which eventually results in recursive dependencies and build errors of the application.

This can be seen in couple of different ways, they are; 

• Recursive dependencies

cabal: Error: some packages failed to install: snap-core-0.9.7.0 failed during the building phase. 

The exception was: ExitFailure 1

• Hidden packages/modules

SnapUtil.hs:26:18:

    Could not find module ‘Safe’

    It is a member of the hidden package ‘safe-0.3.9’.

    Use -v to see a list of the files searched for.

• Uncertainty of the package by which a particular module is being imported

 CMD.hs:19:17:

    Ambiguous occurrence ‘lookup’

    It could refer to either ‘CMD.lookup’, defined at                 CMD.hs:12:20 or ‘Prelude.lookup’,  

         imported from ‘Prelude’ at CMD.hs:3:8-10 (and              originally defined in ‘GHC.List’)

• Could not resolve dependencies, Backjump limit reached

cabal: Could not resolve dependencies:
trying: snap-0.14.0.6 (user goal)
......................
......................
Backjump limit reached (change with --max-backjumps)

To solve them there are numerous ways. Here I'm going to share some methodologies I know to go around the above issues.

• Solution to recursive dependencies

The reason behind recursive dependencies is; registering the same package in both System directory and User directory of cabal. In order to solve this you should take a look at the packages installed with the 'ghc-pkg list' command as follows;

It lists both System directory packages and User directory packages.

System directory packages

User directory packages

 If you can eye the repeating packages you can unregister them using;

'ghc-pkg unregister <packageName | packageName-version>'

• Solution to shadowed packages/ modules

In case of a hidden package you can see in the package list in parenthesis as follows;

Here the packages 'monad-tf-0.1.0.2' and 'network-uri-2.6.0.3' are hidden. You can expose those packages using;

'ghc-pkg expose <packageName | packageName-version>'

In a scenario where there are hidden modules, there is no solid solution in order to use the same module in your code except for switching to another module in a different package, because this is done by the author of the package at the time of publishing it. You can check this using the following command for the specific package, which the module is imported from;

 'ghc-pkg describe <packageName>'

The result of that command for the 'http-client'

 Here, you can see the hidden modules and the exposed-modules from the package. 

• Solution to uncertainty of package by which a particular module is being imported

The best way to solve this is to import packages with the "qualified" tag when you are in doubt whether there are multiple packages importing the methods with the same name. 

Another way is to import the packages as follows;

 'import Data.Aeson as DA'

That way you can always import the methods from the respective package like this,

..............

..... let s = DA.encode json 

............. 

 which will make anyone reading the code to understand it easily as well.

These are some ways, I thought the best ways to solve the dependency problems in Haskell. In any case if you mistakenly unregister a package, you can always install it again by using 

'cabal install <packageName>'

• Solution to Backjump limit reached, which lead unresolved dependencies

A sample of this error, when installing packages in cabal is as follows;

This error occurs when the dependency tree searched exhaustively. One way to solve this is to arrange the dependencies into an order, which will make it easier for the compiler resolve them. To do this we can use,

'cabal install <packageName> --reorder-goals' 

Another solution, as the compiler suggests is to change the 'max-backjumps' from the default to a preferred value. 

Hope this post helped you !!