We are a 4 people-size company and it works just well. We do lots of work and external people often think we are more than that.
Holacracy will certainly bring a lot of value, making everything explicit.
I would suggest to start with only one circle and a couple of roles, no more. Otherwise, it will be overwhelmed.
Would love to hear your feedback once implemented.