There is a fair number of steps involved in the set up of an instrument. And this isn't meant to be a comprehensive "How To" on the subject. But I'll give a couple of pointers that should get you through with acceptable results. In fact, a lot better than many factory instruments that I'm asked to work on.
When making a nut, you need to determine the height of the frets so you have a reference in shaping it. Use a pencil that has been flattened on one side to ride on top of the frets to mark this on the nut.
Once you have removed all the material that isn't required and sanded and polished it up you then need to determine where the strings will be. I use a straight edge to determine this.
After making with a very sharp pencil the two outside string positions and scoring those lines with a fine saw you then you grab your string spacing tool made by Ken Timms.....OK, it's not absolutely necessary, but it works a treat and really cool.
Score those lines with your fine saw and then deepen all 4 slots with your sting files. Not too deep. Just enough to keep the strings in place. We'll adjust depth later.
Determining the action at the 12th fret is also easy to work out with something as simple as a drill bit and a straight edge. Grab a 1/8" drill bit and place it at the 13th fret. Lay your straight edge on top of the first fret and the drill bit. Measure the distance from the bottom of the saddle slot to the bottom of the sttraight edge and it tells you how tall your saddle needs to be for the correct action at the 12th. Here I've already made the saddle and just checking to confirm the measurements.
And finally once the strings are on, you adjust the action at the nut. This is done by fretting the string at the 3rd fret and observing the distance between the bottom of the string and the top of the fret. It should be apporoximately the thickness of us old timers would refer to as a rolling paper. Perhaps a couple of them for the thicker C or low G strings.
You sneek up on this adjustment. As when you get close it can be the difference between one or two strokes of the file as to being spot on or too low.
You may need to adjust the action at the 12 fret again once the strings have had tension on them for a few days. It's common and desirable for the top to pull up and the bridge to rotate a bit and could mean adjustments are required.