Can I only borrow items from the library where I registered?

You can borrow library items from any library across the city, not just the library where you joined.