For the marinade: Grate the zest from the oranges and then squeeze the juice. Measure out 120 ml (4 fl oz) orange juice for the marinade. Grate the zest from the lemon.
Combine the orange zest and juice, lemon zest, chipotles and olive oil in a baking dish or other container that will hold the skewers. Whisk to combine.
Cut the swordfish into 4 cm (1½ in) cubes. Divide into 4 portions and double-skewer them on the bamboo skewers.
Add the skewers to the baking dish, rolling them to coat with the marinade. Make sure to coat with the minced chipotles on all sides of the meat. Cover with plastic wrap and marinate in the refrigerator for at least 2 hours, no more than 4 hours.
Heat a grill pan or prepare an outdoor grill. If using a grill, when the coals are hot, set the grate on the lowest level (closest to the coals) and get it very hot, then brush the grate with vegetable oil.
Grill the fish for 2 minutes per side on all four sides, for a total of 8 minutes.