Skip to main content

· 9 min read

My name is Bartłomiej Cieślar, and I have done a 5-month long internship at IOG. Throughout the internship my work was focused on improving the Glasgow Haskell Compiler. In this post I share a bit about the work that I did and the new features that are available in GHC 9.8 and GHC 9.10 as a result.

Deprecating Exports​

Originally in GHC, the only way to deprecate usages of certain things was to deprecate their definitions, as in the following example:

foo :: Int -> Int
foo x = x * x

{-# DEPRECATED foo "Do not use" #-}

However, consider the following (real) issue: there are several functions in the module Data.List (lines, words, unlines and unwords) that have been moved to Data.String; however, their exports cannot be removed from base, since in doing so it would immediately break any code using those functions through Data.List without any heads-up. This is where the new deprecated warnings come in.

Under the new syntax (available from GHC 9.8), it is possible to write deprecation and warning annotations next to the export definitions:

module Data.List where
       ( {-# DEPRECATED "Moved to Data.String" #-} lines
       , {-# WARNING "Will be removed in the next release!" #-} SomeType(..) )
import Data.String (lines)
import Types (SomeType(..))

There are two corner cases:

  • an identifier imported from several modules leading to inconsistent annotations in the importing module
  • an identifier exported more than once with inconsistent annotations in the exporting module

The first case is illustrated with the following example:

module Good where
foo :: Int
foo = 10
----------------------
module Wrong
  ( {-# DEPRECATED "Moved to Good" #-} foo
  )
where
import Good
----------------------
module Test where

import Wrong (foo) -- warning here since explicitly mentioned in Wrong's import list
import Good

bar = foo       -- no warning here since imported without deprecation from Good
baz = Wrong.foo -- warning here since the import is qualified with Wrong

foo is imported from both Good and Wrong modules but it is deprecated only in Wrong's export list. Warnings are emitted every time the identifier is explicitly used from Wrong:

  • in Wrong's import list
  • when an occurrence is qualified with Wrong
> ghc-9.8 Test.hs

Test.hs:3:15: warning: [GHC-68441] [-Wdeprecations]
    In the use of ‘foo’ (imported from Wrong):
    Deprecated: "Moved to Good"
  |
3 | import Wrong (foo)
  |               ^^^

Test.hs:7:7: warning: [GHC-68441] [-Wdeprecations]
    In the use of ‘foo’ (imported from Wrong):
    Deprecated: "Moved to Good"
  |
7 | baz = Wrong.foo
  |

The second case is illustrated with the following example:

module Both
  ( {-# DEPRECATED "Moved to Good" #-} foo
  , foo
  )
where
import Good (foo)

foo identifier is exported twice with inconsistent annotions in module Both: both with and without a deprecation annotation. A module importing foo from Both won't raise a deprecation warning (there is a non-deprecated export after all). To help avoiding this situation, a new compiler flag -Wincomplete-export-warnings can be enabled (it is included in -Wall) to make the compiler warn about such inconsistent annotations in the defining module:

> ghc-9.8 Both.hs -Wall

Both.hs:2:5: warning: [GHC-94721] [-Wincomplete-export-warnings]
    ‘foo’ will not have its export warned about
    missing export warning at Both.hs:3:5-7
  |
2 |   ( {-# DEPRECATED "Moved to R" #-} foo
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Both.hs:3:5: warning: [GHC-47854] [-Wduplicate-exports]
    ‘foo’ is exported by ‘foo’ and ‘{-# DEPRECATED "Moved to R" #-}
                                    foo’
  |
3 |   , foo
  |     ^^^

References:

Note: The WARNING pragma behaves exactly like the DEPRECATED pragma. In previous examples and in the rest of this post we always use the DEPRECATED pragma but the WARNING pragma could have been used too.

Deprecating Instances​

Similar to exports, there was previously no way to deprecate single instances of type classes. For example, the NFData class is responsible for implementing deep strictness on types and there is an instance of it for functions which does not make much sense and is pretty inefficient:

instance (Enumerate a, NFData b) => NFData (a -> b)

However, it cannot be removed since this would suddenly break all code using this instance, not giving the library users time to update their code.

From GHC 9.10, there will be a way to do add deprecation pragmas to instances:

instance {-# DEPRECATED "Do not use" #-} (Enumerate a, NFData b) => NFData (a -> b)

which will emit a warning every time a type class constraint is solved to this instance. It is also possible to add warnings to derived instances, although they have to be derived with standalone deriving declarations:

deriving instance {-# DEPRECATED "Will be removed soon" #-} Show T

References:

Incomplete Record Selectors​

In haskell, one can name the fields of a data type constructor, in which case that constructor is called a record:

data T = T1 { fieldX :: Int } | T2 Bool | T3 { fieldX :: Int, fieldY :: Char }

In order to access the fields of a record, one can pattern match on it:

foo1 :: T -> Int
foo1 (T1 {fieldX = val}) = val
foo1 _                   = 0

or use a record selector function:

foo2 :: T -> Int
foo2 t = fieldX t + 2

The issue is that a record selector is not defined for the constructors which do not have that field. Such record selectors are called incomplete. In our example, fieldX and fieldY are incomplete record selectors. As a consequence, calling foo2 on a value constructed with constructor T2 would fail at runtime with an exception becaues T2 doesn't have a fieldX field.

There is already a warning -Wpartial-fields that warns about such record fields at the definition site:

> ghc T.hs -Wpartial-fields
[1 of 1] Compiling T                ( T.hs, T.o )

T.hs:3:15: warning: [GHC-82712] [-Wpartial-fields]
    Use of partial record field selector: ‘fieldX’
  |
3 | data T = T1 { fieldX :: Int } | T2 Bool | T3 { fieldX :: Int, fieldY :: Char }
  |               ^^^^^^

T.hs:3:63: warning: [GHC-82712] [-Wpartial-fields]
    Use of partial record field selector: ‘fieldY’
  |
3 | data T = T1 { fieldX :: Int } | T2 Bool | T3 { fieldX :: Int, fieldY :: Char }
  |                                                               ^^^^^^

Note: the message Use of partial record field selector: ... should better be reworded as Definition of partial record field selector: ....

However:

  1. it is only a warning because incomplete record selectors are sometimes desirable.
  2. it is only visible when compiling a module defining partial record field selectors, not when a partial record field selector field is used in some client module.

Therefore, I've implemented a new -Wincomplete-record-selectors warning (available from GHC 9.10) that warns about occurrences of incomplete record selectors that can't be proved not to fail.

foo3 :: T -> Int
foo3 (T1 {fieldX = x}) = x
foo3 t                 = fieldX t -- warning emitted here

foo4 :: T -> Int
foo4 (T2 _) = 0
foo4 t      = fieldX t -- warning not emitted here, since T2 is handled by the previous equation
> ghc-9.10 Test.hs -Wincomplete-record-selectors
Test.hs:7:26: warning: [GHC-17335] [-Wincomplete-record-selectors]
    The application of the record field ‘fieldX’ may fail for the following constructors: T2
  |
7 | foo3 t                 = fieldX t -- warning emitted here
  |

This also works with GADTs for which we need to take types into account to know which constructors are allowed to occur. In the following example, it doesn't make sense to warn in bar2 about handling the case where g is constructed with G2 because g's type prevents it.

data G a where
    G1 :: {fieldZ :: Int, fieldC1 :: Char} -> G a
    G2 :: {fieldC2 :: Char} -> G Int

bar1 :: G a -> Int
bar1 g = fieldZ g -- warning emitted here

bar2 :: G Char -> Int
bar2 g = fieldZ g -- warning not emitted here, since G2 cannot occur
> ghc-9.10 Test.hs -Wincomplete-record-selectors

Test.hs:8:10: warning: [GHC-17335] [-Wincomplete-record-selectors]
    The application of the record field ‘fieldZ’ may fail for the following constructors: G2
  |
8 | bar1 g = fieldZ g -- warning emitted here
  |

References:

Other minor contributions​

For the purpose of this part, let us assume that module Mod has the following definition:

module Mod where
data X = X {x :: Int, y :: Char}
{-# DEPRECATED x "Will be removed soon" #-}

Broken deprecations for record fields​

Since GHC 9.4, deprecation warnings attached to record fields were not emitted when record fields were accessed via record selectors, the usage of HasField, or with the overloaded record dot syntax:

module Mod2 where
import Mod
import GHC.Records
foo :: X -> Int
foo = x -- no warning emitted here (record selector)

bar1 :: HasField "x" t Int => t -> Int
bar1 t = getField @"x" t

bar2 :: X -> Int
bar2 = bar1 -- no warning emitted here (HasField constraint)

baz :: X -> Int
baz t = t.x -- no warning emitted here (record dot syntax)

I can happily say that with the help of Sam Derbyshire and Simon Peyton-Jones the bug was fixed and this feature will work again fully in GHC 9.8.

References:

Custom deprecations in record wild card syntax​

In GHC, one can pattern match on all fields of a record using the record wild card syntax, which will assign the value of the all remaining record fields to the variables with the same name:

import Mod
foo :: X -> Char
foo (X {..}) = y

however, when using this syntax, custom deprecations of record fields won't be emitted even if those fields are deprecated:

bar :: X -> Int
bar (X {..}) = x -- no warning

this was because if there were a lot of fields deprecated in the pattern match, the warnings would have been noisy. However, since GHC 9.10 it will be changed so that the custom warnings for record fields will be emitted only if the variables to which their values are assinged to are used in the body of the function.

References:

· 16 min read

Banner

Introduction​

As the Cardano ecosystem continues to grow and evolve, contributors to the Cardano ecosystem are committed to continually refining and optimizing Cardano's networking infrastructure. The release of Dynamic peer-to-peer (P2P) networking, delivered with node v.1.35.6, was a collaborative effort of the networking team from [IOG], [Well-Typed], [PNSol] and the [Cardano Foundation] and represents a highly performant deliverable and a significant milestone in Cardano's journey toward establishing a fully decentralized and secure blockchain platform.

Given that Cardano functions as a real-time stochastic system, its performance and security are inherently interconnected. The networking team remains committed to finding the ideal balance among various factors, including topological and topographic considerations, to enhance timeliness and connectivity.

This blog post takes you through the engineering journey behind the development of Cardano's Dynamic P2P design. It delves into the core design principles, highlights the challenges encountered along the way, and unveils the solutions the team devised to establish a robust and scalable networking system.

What is Dynamic P2P​

The Dynamic P2P implementation continuously and dynamically refines the active topology through a peer selection process, with the objective of reducing the overall diffusion time across the entire network. Research findings suggest that utilizing a policy based solely on local information can result in an almost-optimal global outcome. This is achieved by monitoring the timeliness and frequency of peers that provide a block header, which is ultimately incorporated into the chain.

The primary goal is to eliminate highly ‘non-optimal’ peers while maintaining strong connectivity. To achieve this, peers considered less useful based on this metric are periodically ‘churned out’ and replaced with randomly selected alternatives. Simulation results indicate that this optimization method converges towards a near-optimal global outcome within a relatively small number of iterations.

Practically, Dynamic P2P replaces the manual configuration of peer selection (e.g. using the topology updater tool).

With manual configuration, stake pool operators (SPOs) were required to establish connections with a significant number of peers (50 for example) to maintain a minimum of 20 active connections consistently. This approach was necessary due to the static nature of configured peers and the varying availability of SPO relays.

However, with Dynamic P2P, nodes can be configured to maintain a specific number of active peer connections (e.g. 20) and select from all registered SPO relays on the chain. In the event of a lost connection with a peer, the node will automatically select alternative peers and persistently attempt connections until the desired target is reached.

As a result, Dynamic P2P eliminates the requirement for over-provisioning of connections, offering a more efficient and adaptable networking solution.

· 19 min read

Introduction​

I recently gave a short presentation on the topic of stacks in the GHC JavaScript backend to the GHC team at IOG. This blog post is a summary of the content.

In the context of a program produced by the GHC JavaScript backend, two different types of stack exist: The JavaScript call stack and Haskell lightweight stacks. In this post we will focus mostly on the lightweight stacks.

First we will see why using only the JavaScript call stack is not suitable for running compiled Haskell code. Then we will introduce the calling convention we use for Haskell and see how the lightweight stacks are used for making calls and passing around data. After this, we will explore in more detail how they are used for exception handling and multithreading.

· 16 min read

IOSim on Hackage​

The IOG Networking Team is pleased to announce that we published [io-sim], [io-classes], [si-timers], [strict-stm], [strict-mvar] and [io-classes-mtl] on Hackage. These are tools without which we could not imagine writing a complex distributed system like [Cardano].

These packages support our goal of using the same code to run in production and simulation, what greatly increases the reliability and quality of the final system. [io-sim] and its ecosystem is designed to let write a simulation environment which provides provided things usually provided by an operating system like networking stack or disk IO and develop as well as implement & model complex applications/systems.

For developing a robust system one needs a proper testing framework which allows one to model the key characteristics of the system. To achieve this goal we needed to create an abstraction that captures the key aspects of the Haskell runtime and operating system environment for distributed systems. The Cardano [network stack][ouroboros-network] is a highly concurrent system, and as a network application, it needs to deal with time: there are all sorts of timeouts that guard resource usage: inactivity timeouts, message timeouts, or an application level TCP's WAIT_TIMEOUT among others. The tools which we provide permitted us to capture issues related to timing (which abound in network programming) which, in production, would be extremely rare (things like simultaneous TCP open or critical race conditions) and ensure that we can test (in the simulation) these scenarios. Recently we caught a [bug][sim-tcp-open-bug] in simultaneous TCP open when one side of the connection crashed - a corner case of a corner case, that's how effective is the combination of quickcheck style property-based testing & simulation!