Project and package organisation in Scala (and Java)

(, en)

Scala applications can become a mess if you don’t pay attention. Over time, I’ve settled into a few patterns that help keep your project layout clean and maintainable.

The foundation of this post is based on

  1. Global organisation, local chaos » package structure is global organisation, what you do inside a class or object depends on the problem at hand. You can also call it »global consistency, local flexibility«
  2. Community standards trump home-brewed standards » If you decide to use snake case (a_variable_name) for all your variables in a Java project and try to enfoce consistency in your project, you make it difficult for everyone: junior developers, senior developers and new team members.

Let’s start.

Introduction

Most applications follow a 3-layered approach:

  1. Application (app) Layer
  2. Business/domain logic (core) layer
  3. Database (db) layer
       .---------------------------.
       |                           |
       |     Domain/Core Layer     |
       |                           |
       .---------------------------.
             ^               ^
             |               |
             |               |
.----------------.       .----------------.
|                |       |                |
|   App Layer    |- - - >| Database Layer |
|                |       |                |
.----------------.       .----------------.

Even with a hexagonal architecture, you can roughly categorise the parts into the three layers mentioned above.

With this model in mind, two dimensions for package organisation for a (micro-)service become evident:

  1. the (sub-)project or file-based structure
  2. the package (name) of a class/object/trait/…

1. Set boundaries through sub-projects

Having the same package doesn’t mean that you have to put the files in the same (sub-)project. In Scala, you can split your application into

The naming can vary, some like to name db as database or api as api-description. This is not so important, more important is the split.

Why? With this split you can enforce layer boundaries:

Also testability benefits from this split

I have some remarks on this structure, mostly because if you want to do it perfect, you will fail or at least spend a lot of time on trying to get the project setup right. My conclusion: accept the »Law of Leaky Abstractions«

All non-trivial abstractions, to some degree, are leaky. – Joel Spolsky

Remarks on Domain and Database Layers

In Domain Driven Design (DDD) it is common to separate database from the domain/core layer, but in practice they tend to get coupled.

Performance requirements are pushing the domain logic closer to the database layer. This is expecially true for bulk operations. In the end you are on a network, no abstraction will abstract this away. Reducing round-trips across the network is a low hanging fruit in performance optimalisation.

That said:

Premature optimization is the root of all evil – Donald Knuth

If used with care and good test coverage, shifting domain logic into the db layer is a very powerful approach. Martin Fowler’s case study covers this as well.

Why keep them in separate sub-projects then?

Remarks on Application vs. Domain Services

In DDD circles you can also find the distinction between application and domain services. If find this distinction overly academic. Services are services.

2. Package Organisation

Many projects in Scala (and Java) suffer from »Categoritis«

Categoritis is a tongue-in-cheek term (often used humorously or critically) to describe the overuse or misuse of categories, types, or abstractions in software design - especially when it becomes counterproductive. – ChatGPT

The result: a very fine-grained package structure with one class per package.

But what is the benefit? Consistency?!?

Honestly I have no clue. In my time as developer I had many discussions about consistency. I always come back to this StackOverflow thread:

Consistency is of paramount importance. …but it comes with a caveat… You and you coworker are likely obsessing over the wrong sort of consistency – nsfyn55

So the question is: what is de return on investment, given that consistency is quite expensive and difficult to maintain?

Of course consistency has its benefits, but the biggest draw-back is that you are restricting yourself to a smaller solution space and open the doors for unneeded complexity.

I try to be consistent at the top (package organisation), relax at the bottom (the implementation level).

This results in a package structure that is sufficient in 99.99% of the cases:

org.bargsten.<system>.<service>.<feature>.core          // <--- here you can put the services
org.bargsten.<system>.<service>.<feature>.api
org.bargsten.<system>.<service>.<feature>.api.dto
org.bargsten.<system>.<service>.<feature>.kafka
org.bargsten.<system>.<service>.<feature>.db

Example

A microservice, part of a banking system, handling transactions packages could be named:

org.bargsten.banking.transaction.swift.*
org.bargsten.banking.transaction.sepa.*

More Granular Package Structure

If you want to have a finer-grained distinction, you can split .core further:

org.bargsten.<system>.<service>.<feature>.core.service   // <--- here you can put the services
org.bargsten.<system>.<service>.<feature>.core.model
org.bargsten.<system>.<service>.<feature>.core.error
org.bargsten.<system>.<service>.<feature>.api
org.bargsten.<system>.<service>.<feature>.api.dto
org.bargsten.<system>.<service>.<feature>.kafka
org.bargsten.<system>.<service>.<feature>.db

For versioned APIs, add a version suffix:

org.bargsten.<system>.<service>.<feature>.api.v3
org.bargsten.<system>.<service>.<feature>.api.v3.dto

Alternative considerations:

org.bargsten.<system>.<service>.<feature>.db    -> org.bargsten.<system>.<service>.<feature>.core.db
org.bargsten.<system>.<service>.<feature>.kafka -> org.bargsten.<system>.<service>.<feature>.infra

Pick what you like!

Help from IntelliJ

IntelliJ has a »package« view, allowing you to group by package name. I use this a lot, showing the domain repository trait (in .core) together with the implementation (in .db), even though they are in a different subproject.

This way you can enforce the separation of domain/core from the rest.

IntelliJ Project View IntelliJ Package View

IntelliJ Packages View Setup

Conclusion

Project organisation and package naming are hard. But it’s worth figuring out what makes sense in your project.

And remember: keep it simple.

Does this post resemble the well known Hexagonal Architecture? I think not. It is more relaxed.

I dumped the Scala examples in a git repo on Github

See Also