Can my ex-landlord do this?

This is a bit of a long story, so please forgive me.

Last April ((2024)) I rented from a landlord in my area. He explained that the apartment was legal, but asked that we pay in cash, and that we don't report it on taxes. I agreed but not sure why it really mattered one way or another. Payments were in cash, and were done on the 1st of the month. He asked for first and last on the day we planned to move in, but ended up getting it early. We never ended up signing anything. We later found out he doesn't report this income to the CRA.

My partner and I moved in, and 6 months later found a home we couldn't pass up on. We put an offer on the home and it was sold to us. We let our ex-landlord know what was up and that we were moving out. We thanked him for his residency and apologized that it was so soon (we gave 45-30 days notice). He was upset and told us we had to fill the unit. We gave him 2 leads that fizzled out for reasons that were beyond our control. He hasn't posted or otherwise searched for other tenants at this point, and remains that it's our responsibility to find him new tenants.

We have lived in our new home now for a couple months and he has hunted my family down at their place of work, called us a bunch of times, and shown up to our new residence. We never told him where we moved to, and we are almost an hour away from the rental home. He is requesting we pay him the rest of our rent we owed him. He has told us if we don't pay, he will be taking us to court. He also told us to leave the bed frame there, and then demanded us to take it away from the unit. When we got there is when he told us we should pay him or he will take us to court.

What I don't know is if he is able to actually do this? He never registered the apartment with LTB or had us listed as tenants anywhere. We didn't have a set tenancy agreement, and he didn't record the income he made on his taxes. Ontop of this, the bathroom had mould, and I was newly pregnant at the time and had a hard time breathing. He also as I said, has spammed me, my partner and my family (dad - emergency contact) for months now, as well as showing up to our new house uninvited.

Thank you for any advice.