I've read to use wood glue on the rough edges then lightly sand it. I tried it for a project and it didn't work for me.
I've never made a speaker from scratch, been thinking of trying for a simple sound bar for the wife and daughter's TV. Those tiny 3/4" speakers grind my ears.