ず

ず
Verb ない stem + ず
する→せず
くる→こず

The ず form of a verb has the exact same meaning as the ないで form, which means without doing.
The ず form of a verb will always be paired with adverbial に, which fills the role of で in ないで.

If there’s a difference, its that ずに is shorter and quicker to say than ないで, and from my experience seems to be more common. It’s also preferred in writing due to it being an older form. 
Its also used for certain ず expressions. 

Examples

• 何も食べずに困っている – hasn’t ate anything and is having trouble.
• 何も見ずに絵を描く – Draw a picture without looking at anything. 
• 傘を持たずに出かけた – Went out without an umbrella. 

ずにすむ

Ended Without having to do X
ず form + にすむ

ずにすむ means that something finished without you having to do something else.
It is implied that not having to do it was a good thing. 

*すむ here is 済む, to finish/end

Examples

• 宿題をせずにすんだ – It ended without us having to do homework.
• 先生を怒らずに済んでよかった. – I’m glad it ended without angering the teacher.
• 殺されずに済んでよかった. – I’m glad it ending without (me?) being killed. 

ずにはいられない

Can’t help it
ず form + にはいられない

ずにはいられない literally means “Unable to be without doing”. It’s used with a “can’t help it” meaning.
This pattern implies a psychological fault/weakness. 

Examples

• かわいいものを買わずにはいられない – I just can’t help myself but to buy cute things.
• 彼女を見たら笑顔をせずにはいられない – I can’t help but smile when I look at her.
• この写真をみたら、思い出さずにはいられない – When I look at this picture, I can’t help but remember (something). 

Alternate Forms

ずにすむ → ないですむ
ずにはいられない → ないではいられない

Both of the above patterns are apparently ok with substituting ないで for ずに, but they are basically never used in my experience or the guys on stack exchange.
Just be aware that you may see them.