Ion implanted amorphous silicon (a-Si) shows signs of recovery upon annealing that translate into heat release and decrease of bond angle distortion. Yet, little is known about implantation damage structure and evolution in this material. We investigate this process for few keV ions using molecular dynamics, and compare it to damage annealing in crystalline silicon (c-Si). For the simulations, we use a Stillinger-Weber potential properly modified with the two-body Ziegler-Biersack-Littmark potential to account for short-range interactions. The simulation box includes ~10⁵ atoms in both cases and 3 keV ions impinge the surface at 5-13° of incidence. In a-Si, the resulting defects are identified using a method based on the deviation of different parameters from their usual distributions. As a reference, the method is also used in c-Si and compared to a topological identification method. In c-Si, the implanted ions (few keV) cause compact amorphous-like regions to form, while no evidence for melting is observed. The annealing starts by a rapid recrystallization of small clusters and isolated atoms at the surface of the amorphous-like regions (first few ps), then shifts to a long period of sudden annealing steps often involving several neighboring atoms. In a-Si, the initial annealing process is comparable to that in c-Si, but on longer timescales, the defects seem to flow through the rest of the material, tending to distribute the damage uniformly over all the accessible degrees of freedom of the system.