We search for gravitational-wave signals associated with gamma-ray bursts (GRBs) detected by the Fermi and Swift satellites during the second half of the third observing run of Advanced LIGO and Advanced Virgo (2019 November 1 15:00 UTC–2020 March 27 17:00 UTC). We conduct two independent searches: a generic gravitational-wave transients search to analyze 86 GRBs and an analysis to target binary mergers with at least one neutron star as short GRB progenitors for 17 events. We find no significant evidence for gravitational-wave signals associated with any of these GRBs. A weighted binomial test of the combined results finds no evidence for subthreshold gravitational-wave signals associated with this GRB ensemble either. We use several source types and signal morphologies during the searches, resulting in lower bounds on the estimated distance to each GRB. Finally, we constrain the population of low-luminosity short GRBs using results from the first to the third observing runs of Advanced LIGO and Advanced Virgo. The resulting population is in accordance with the local binary neutron star merger rate.