Bhat's Notes

Excerpts from the Fireside Chat between Prof. PK and Manohar Paluri, VP of AI at Meta, at IIITH on 3rd Jan 2025.

About Manohar Paluri: Returning to one’s alma mater to deliver a talk is always a fulfilling experience. Manohar Paluri, an alumnus who graduated from IIIT Hyderabad 22 years ago shared his insights on the future of AI, focusing on end-to-end systems and inference. His happiness was evident as he addressed the current students, reflecting on the advancements in AI and the increasing importance of integrated systems and efficient inference mechanisms.

About Prof. PK: If you envision a scientist as reserved and introverted, Professor Ponnurangam Kumaraguru, affectionately known as “PK” from IIIT Hyderabad, will challenge that stereotype. Naturally outgoing and active on social media, Professor PK thrives in academia, relishing the freedom to select research problems that captivate him. For example, today he gave the students an idea on how to jailbreak the guard rails of hacking! Of course it is known secret but it is his sportive nature to discuss this.

Prof. PK: Building first in the race?

Manohar: Who remembers who landed on the moon second? Everyone competes, but being first is just as important as being the best. It’s all about striking the right balance between the two.

Prof. PK: You mentioned having full privacy with glasses—could you clarify what you mean by that?

Manohar: Privacy has two key aspects. First, knowing what’s happening—having clarity and control. Second, transparency, which builds trust and confidence. It’s similar to how KYC works in India. Without these elements, the system starts to break down.

Prof. PK: You mentioned systems built around the community. How do you define “community”?

Manohar: After I transitioned from research to working on Llama, which blends 50% engineering and 50% research, I took a trip across Asia. During that trip, I noticed how different countries were customizing Llama to meet their unique needs. For me, community is about building on top of Llama and giving back to improve it further. We need more contributors than consumers. This principle is what made PyTorch so successful—it thrives on a culture of giving and collaboration.

Prof. PK: How many people in the audience here are using Llama?

15% audience polled.

PK: How small is small going to be?

Manohar: Take a mixture of experts architecture—it can be up to 254 times larger than the number of active parameters. But are we talking about active parameters or the total number of parameters? Inference cost and the cost of running on a device are far better metrics than just counting parameters. For example, with smart glasses, live translations can happen directly in your ears, allowing you to speak naturally in your own language—almost like The Hitchhiker’s Guide to the Galaxy.

PK: Bias in the generated inage of a doctor!

Manohar: When a query is processed, it gets rewritten based on the underlying data distribution. For example, an image of a doctor might be translated or reinterpreted with considerations for biases and fairness. That’s why you often see different answers. After all, even in society, we haven’t universally agreed on a single perspective or answer.

Manohar: When a query is processed, it gets rewritten based on the underlying data distribution. For instance, an image of a doctor might be interpreted or adjusted with considerations for biases and fairness. This is why you may encounter varying answers. After all, even in society, we haven’t reached a universal consensus on a single perspective or truth.

Prof. PK: If WhatsApp already knows your phone number and, by extension, your region, would it then be acceptable to have a bias and display the image of an Indian doctor for users in India?

Manohar: This is fundamentally a policy question. Sitting in Silicon Valley and assuming we can solve a city’s problem from afar isn’t always effective. What we need is culture-specific image generation, where users have the power to customize outputs based on their context and preferences. That approach is far more powerful and impactful.

Prof. PK: Are you referring to the bias of an annotator?

Manohar: Whenever there’s a human in the loop, some degree of human bias is inevitable.

Prof. PK: Is there any app you’d like to share with us?

Manohar: It’s The Clown. Have you heard of it? It’s a custom puzzle experience. I never imagined I’d have such an intense interaction with an AI. I spent 20 minutes talking to it, and honestly, it was quite scary.