170k loan
(1) Credit Union: 4.325% with a 1% origination fee and another $300 for processing, so costs directly to credit union are $2,000 approximately.
(2) Mortgage broker: 4.50% with $1,425 of fees for the broker rolled into the loan.
Or doesn’t it make much difference? Thanks for your advice and knowledge.

It can depend on how long you stay in the house.
The first option is better long term
Second option better for short term
Reasons: Option 1 costs more in fees, but the lower rate calcs to a lower monthly payment by around $15 a month (assuming fees are rolled in the new balance in both cases) It will take 38 months for that $15 a month to “pay for” the $575 in extra fees. From that point on, you are better off, so it’s better long term.
If you plan to move inside of that 38 months, option 2 would be better between the two, however, it may be better to not refinance at all in that case and avoid the fees altogether. You would need to compare option 2 with your current mortgage over various terms to see if you would really benefit with a refi.
Also, if you might move soon, it might also be better to do nothing. The fees to refi will eat up any savings on the lower rate if you end up moving. I am living this first hand. I refi-ed last year and dropped the rate from 5.75 to 4.875, lowered my payment by around $100. But it costs $2400 to do it in fees. So it takes 24 months at the new payment savings just pay off that $2400 in fees. I turns out I am looking at moving right now, so if I sell only 14 months later, I really paid $1000 for nothing.
Think it through.
There are a few perfect suggestions on this topic. check out the suggestions on http://www.e-loansresources.tk