Until I found TKL I used to make my own "speciality" images with T2 linux.
Basically it only installs the packages you require and results in an extremely lean system.
If you have a look at askozia.com (a free PBX solution) they have some amazing functionality in a 20Mb package. I think the next version might blow out to a whopping 30Mb! They have used T2 ever since porting from FreeBSD to Linux.
Is there a reason for using Ubuntu or Debian as the base for TKL or would there be benefit in using a "distribution build kit" like T2?