Depending on the jurisdiction, always have a baseball and potentially gloves with you as well. Having only a baseball bat can be considered as either possession of a weapon or having an intent to cause harm I think, which can really backfire when authorities get involved.

IANAL, and anyone reading should consult their local laws surrounding self-defence before carrying items for the express purpose of self-defence