Project and package organisation in Scala (and Java)
Scala applications can become a mess if you don’t pay attention. Over time, I’ve settled into a few patterns that help keep your project layout clean and maintainable.
The foundation of this post is based on
- Global organisation, local chaos » package structure is global organisation, what you do inside a class or object depends on the problem at hand. You can also call it »global consistency, local flexibility«
- Community standards trump home-brewed standards » If you decide to use snake case
(
a_variable_name) for all your variables in a Java project and try to enfoce consistency in your project, you make it difficult for everyone: junior developers, senior developers and new team members.
Let’s start.
Introduction
Most applications follow a 3-layered approach:
- Application (app) Layer
- Business/domain logic (core) layer
- Database (db) layer
.---------------------------.
| |
| Domain/Core Layer |
| |
.---------------------------.
^ ^
| |
| |
.----------------. .----------------.
| | | |
| App Layer |- - - >| Database Layer |
| | | |
.----------------. .----------------.
Even with a hexagonal architecture, you can roughly categorise the parts into the three layers mentioned above.
With this model in mind, two dimensions for package organisation for a (micro-)service become evident:
- the (sub-)project or file-based structure
- the package (name) of a class/object/trait/…
1. Set boundaries through sub-projects
Having the same package doesn’t mean that you have to put the files in the same (sub-)project. In Scala, you can split your application into
app, containing- infrastructure stuff
- api stuff
- kafka stuff I implicitly assume that you have a message bus (usually Kafka). Every message and event you publish on a Kafka topic is part of your (external) API. This is why Kafka-related code is part of the API layer.
api(optional) for separating your API description from the implementation, which resides inappdb(database)core(ordomain)integration(for integration tests)
The naming can vary, some like to name db as database or api as api-description. This is not so important, more important is the split.
Why? With this split you can enforce layer boundaries:
- the database layer depends on the domain layer
- the app layer depends on the domain layer
- the app layer (indirectly) depends on the database layer: it instantiates the services/repositories of the database layer.
- the domain layer is dependency free
Also testability benefits from this split
- you can test the database layer independent of the app layer.
- you can test the app layer without the database layer
- your domain layer can be tested without any additional dependencies
I have some remarks on this structure, mostly because if you want to do it perfect, you will fail or at least spend a lot of time on trying to get the project setup right. My conclusion: accept the »Law of Leaky Abstractions«
All non-trivial abstractions, to some degree, are leaky. – Joel Spolsky
Remarks on Domain and Database Layers
In Domain Driven Design (DDD) it is common to separate database from the domain/core layer, but in practice they tend to get coupled.
Performance requirements are pushing the domain logic closer to the database layer. This is expecially true for bulk operations. In the end you are on a network, no abstraction will abstract this away. Reducing round-trips across the network is a low hanging fruit in performance optimalisation.
That said:
Premature optimization is the root of all evil – Donald Knuth
If used with care and good test coverage, shifting domain logic into the db layer is a very powerful approach. Martin Fowler’s case study covers this as well.
Why keep them in separate sub-projects then?
- Testability: dealing only with the database layer (and perhaps the domain layer) is considerably easier
- Discoverability: new developers are used to have this setup and get accustomed faster
Remarks on Application vs. Domain Services
In DDD circles you can also find the distinction between application and domain services. If find this distinction overly academic. Services are services.
2. Package Organisation
Many projects in Scala (and Java) suffer from »Categoritis«
Categoritis is a tongue-in-cheek term (often used humorously or critically) to describe the overuse or misuse of categories, types, or abstractions in software design - especially when it becomes counterproductive. – ChatGPT
The result: a very fine-grained package structure with one class per package.
But what is the benefit? Consistency?!?
Honestly I have no clue. In my time as developer I had many discussions about consistency. I always come back to this StackOverflow thread:
Consistency is of paramount importance. …but it comes with a caveat… You and you coworker are likely obsessing over the wrong sort of consistency – nsfyn55
So the question is: what is de return on investment, given that consistency is quite expensive and difficult to maintain?
Of course consistency has its benefits, but the biggest draw-back is that you are restricting yourself to a smaller solution space and open the doors for unneeded complexity.
I try to be consistent at the top (package organisation), relax at the bottom (the implementation level).
This results in a package structure that is sufficient in 99.99% of the cases:
org.bargsten.<system>.<service>.<feature>.core // <--- here you can put the services
org.bargsten.<system>.<service>.<feature>.api
org.bargsten.<system>.<service>.<feature>.api.dto
org.bargsten.<system>.<service>.<feature>.kafka
org.bargsten.<system>.<service>.<feature>.db
Example
A microservice, part of a banking system, handling transactions packages could be named:
org.bargsten.banking.transaction.swift.*
org.bargsten.banking.transaction.sepa.*
More Granular Package Structure
If you want to have a finer-grained distinction, you can split .core further:
org.bargsten.<system>.<service>.<feature>.core.service // <--- here you can put the services
org.bargsten.<system>.<service>.<feature>.core.model
org.bargsten.<system>.<service>.<feature>.core.error
org.bargsten.<system>.<service>.<feature>.api
org.bargsten.<system>.<service>.<feature>.api.dto
org.bargsten.<system>.<service>.<feature>.kafka
org.bargsten.<system>.<service>.<feature>.db
For versioned APIs, add a version suffix:
org.bargsten.<system>.<service>.<feature>.api.v3
org.bargsten.<system>.<service>.<feature>.api.v3.dto
Alternative considerations:
org.bargsten.<system>.<service>.<feature>.db -> org.bargsten.<system>.<service>.<feature>.core.db
org.bargsten.<system>.<service>.<feature>.kafka -> org.bargsten.<system>.<service>.<feature>.infra
Pick what you like!
Help from IntelliJ
IntelliJ has a »package« view, allowing you to group by package name. I use this a lot,
showing the domain repository trait (in .core) together with the implementation (in .db), even though they
are in a different subproject.
This way you can enforce the separation of domain/core from the rest.


Conclusion
Project organisation and package naming are hard. But it’s worth figuring out what makes sense in your project.
And remember: keep it simple.
Does this post resemble the well known Hexagonal Architecture? I think not. It is more relaxed.
I dumped the Scala examples in a git repo on Github
