In audio, Jefferson says he broke up with his daughter in defense of cannabis