Generally, when I use the spackling I don't put it on very thin. I tend to pack it inside the corrugation and in the spaces between the pieces. Then after it sets, I am able to sand smooth if necessary prior to priming and then painting and texturing. The spackling tends to get fairly thick and the primer soaks in pretty well. This process tends to produce a pretty durable piece.
Some of my pieces are ten years old. Now I do tend to take care of my stuff and my terrain is no exception. UV has aged the color of the green turf stuff I use, and there are dents in some of the pieces where they have been abused, but I can always touch them up if I feel the urge, but so far I have never done so.
I do have a large dungeon in seven pieces that is beginning to show age with dents, dings, corners separating and bits of plaster that has chipped off. That is going to be repaired if I find a buyer but it was my first piece and has been through about eight conventions. Lots of abuse moving it about and players banging on it. Still it is looking pretty good.