Depends. If it's a partisan election, then no, they wouldn't have a choice but to resign once they start running as federal employees (other than an incumbent President or Vice President) are prohibited by the Hatch Act from running as candidates in partisan elections. However, if it's a nonpartisan election (that is, an election - say, for local government - where nobody can be nominated &/or elected while representing a political party), then yes, they could run without resigning.
I was hoping that Hilda Solis would be a good example, but she left the Labor Department over a year before being elected to the LA County Board of Supervisors.
