Many extensions of the Standard Model posit the existence of new heavy particles with long lifetimes. Results are presented of a search for such particles, which decay at a significant distance from their production point, using a final state containing charged hadrons and an associated muon. This analysis uses a data sample of proton-proton collisions at $\sqrt{s}= 8$~TeV corresponding to an integrated luminosity of \lumi collected in 2012 by the ATLAS detector operating at the Large Hadron Collider. Results are interpreted in the context of $R$-Parity violating supersymmetric scenarios. No events in the signal region are observed, with an expected background of $0.02 \pm 0.02$ events, and 95\% confidence level limits are set on squark pair production processes where the squark decays into the lightest supersymmetric particle (the neutralino) followed by the decay of the neutralino into a muon and quarks, and for neutralino lifetime $\tau$ between 3~ps and 3~ns ($c\tau$ between 1~mm and 1~m). To allow these limits to be used in a variety of models, they are presented for a range of squark and neutralino masses.