MIA Meetup 2023: a meeting for women in generative AI

As the gray chairs in the colorful auditorium of Nubank’s Headquarter 1 in São Paulo were filled, a beautiful scene was being built: a presence almost entirely made up of women.

And not just in the audience, but also in prominents positions as speakers and organizers who made the Women in Artificial Intelligence (MIA is the acronym in Portuguese) 2023 event happen, this time in a hybrid format.

MIA is a group born out of the meeting of Women in Data Science (WiDS) ambassadors in Rio de Janeiro and São Paulo, whose mission is to develop and foster a global community that empowers women through the exchange of knowledge in AI, as well as carrying out mentoring work.

The event featured lectures about people management, with Diandra Kubo, Data Science Manager at Nubank; evolution of Large Language Models (LLMs), with Mirelle C. Bueno, Researcher in Natural Language Processing at the Instituto de Pesquisa Eldorado; and opportunities and risks of generative AI, with Kizzy Terra, Founding Partner of the Programação Dinâmica group.

Want to know everything that happened at the event and understand the discussions in the three lectures? Read on!

Choosing the path of People Management

Diandra Kubo, the Data Science Manager at Nubank, opened her talk discussing a critical crossroads many senior Data Science professionals face: Should they remain as a technical specialist or transition into a people management role?

Nubank adopts Y career development plans… professionals must choose to follow the path of a specialist or that of people management.

The idea for the talk came from a text written by Matheus Facure, Staff Data Scientist at Nubank, who didn’t identify with a career in management. Diandra, on the other hand, found herself in people management and brought another perspective.

She confessed her initial hesitations:

1. Could she be responsible for other people’s careers?
2. Was her role to be a specialist or to have a reasonable knowledge of several areas?
3. Would transitioning into management mean abandoning her technical roots?

However, after introspection, she realized her capability to be a multifaceted professional and that she could still be deeply technical while managing people. She decided to take on the challenge and embrace her role as a Data Science Manager.

Challenges in transitioning

Diandra candidly shared the challenges she faced in her new role. She emphasized the importance of:

• Learning to delegate tasks effectively.
• Evaluating not just deliverables but also the overall performance of team members.
• Maintaining unbiased evaluations.
• Maximizing the potential of the team’s talents, including your own, to achieve the greatest impact.

You have to be very careful not to fall into micromanagement and identify what is scalable within the team’s reality.

Tools for success in People Management

Diandra highlighted key strategies and tools that have been instrumental in her journey:

• Active listening: understanding the motivations of the team.
• Radical candor: creating a safe and honest communication environment.
• Aligning expectations: clear objectives and directions.
• Career sessions: ensuring team members are aware of their career trajectory.
• Opportunities: creating moments for team members to exceed expectations.
• Recognition: celebrating strengths and results to boost team confidence.

Gender disparity in Data Science

Towards the conclusion, Diandra addressed a pertinent issue: the need for more women in leadership roles in Data Science.

Female professionals are often asked to do things that would never be considered part of a man’s scope of work.

She emphasized the unfortunate stereotype women face and the need for change in the industry.

Audience Q&A

In the interactive Q&A session, Diandra shared that balancing her technical role with her managerial duties was facilitated by Nubank’s inherently technical nature.

Working at Nubank is a privilege, because it’s a technical place… even when she’s not writing code, she’s still very technical in her work.

When asked about Nubank’s inclusive hiring process, she emphasized:

Nubank does not filter out any college, degree or educational institution in its selection processes. The processes are designed to evaluate based only on the process, and not on academic background.

Diandra Kubo’s talk not only offered valuable insights for those on the cusp of a career transition but also shed light on important industry issues.

Mirelle Candida Bueno, a prominent Researcher in NLP, provided insights into the foundational principles of Large Language Models (LLMs). She emphasized:

Everything behind it [ChatGPT] was already in place. What was missing was organizing everything into a pleasant interface.

Tracing back over 50 years of history, she highlighted that the underlying technology, despite seeming recent, has deep historical roots.

The essence of Language Models

Natural language models operate on the core idea of predicting the next word in a sentence based on the preceding context. A seemingly straightforward concept, but one that took till 2013 to function effectively. She describes these models as a “game of dice” where predictions are conditioned on context and accumulated knowledge.

The early models

The objective of the earliest language models was to automate probabilistic analysis. Their intent was to assess errors and impose penalties. The evolution from simple probabilistic models to today’s automated systems has been long and marked with significant milestones.

Mirelle used a 2D representation to elucidate the architecture of these models. This visualization underscored how similar words reside on the same latent plane, while distinct contexts are farther apart.

The Bi-LSTM and ELMO era

2016 saw the rise of deep learning which catalyzed the development of Bi-LSTM and ELMO models. These models excelled at discerning the most pertinent content contextually. However, Mirelle noted that they “never became extraordinary” due to their issues with memory loss and challenges in fine-tuning.

The Transformer revolution

The term “Transformers”, in the world of Deep Learning, pertains to a groundbreaking architecture introduced in 2017. Distinct from their cinematic counterparts, these Transformers have transformed the way we think about LLMs.

Comprising two layers — one to encode text into vectors and another to generate text — this architecture introduced the pivotal attention mechanism. This ensured that even long commands retained context, addressing the memory loss issue present in earlier models.

Rise of the GPT and BERT models

GPT (Generative Pre-trained Transformer) made its debut in 2017. Despite its humble beginnings, its design principle persists in its successors like GPT-3.5 and GPT-4.

2018 introduced BERT, a model known for predicting masked parts of sentences. Mirelle highlighted its efficiency, stating it doesn’t require extensive data for training unlike its GPT counterparts.

Modern LLMs and their implications

Mirelle expressed concerns about contemporary models like GPT-3 and GPT-4 becoming increasingly proprietary. 

By hindering access to information, these models also hinder research and development.

However, she also spotlighted other noteworthy LLMs:

• LaMDA: the pioneer in dialog data training.
• Chinchilla: a model demonstrating that efficient performance doesn’t necessitate billions of parameters.
• PaLM: which laid the groundwork for Google’s Bard.
• LLaMa: a commendable open-research model from Meta.

Concluding Thoughts

Mirelle concluded by stressing the importance of openness in LLM research. To those keen on delving deeper into the realm of LLMs, she recommended comprehensive courses at institutions like Stanford and platforms like Coursera.

Kizzy Terra’s lecture at MIA 2023 dove into the ethics, risks, and opportunities surrounding the emerging field of Generative AI. Drawing parallels with historical innovations and posing critical questions, Terra explored the potential transformative impact of this technology on society.

AI’s place in history

Drawing from the past, Terra likened the rise of artificial intelligence to the Gutenberg Press of 1430. This comparison underscored AI’s potential to be just as disruptive as the press that made books and documents widely accessible.

Artificial intelligence is just as disruptive as the printing press.

Opportunities of Generative AI

1. Consuming new applications

Generative AI has birthed a myriad of applications like ChatGPT and Midjourney. Terra highlighted the ongoing academic discussions about the use of AI in producing academic papers.

It’s not about ignoring AI’s rampant use, but having technical discussions to define the best way to use them.

2. Creating new applications

With the availability of Large Language Models public APIs, there’s an opening for innovators to build tools that increase accessibility, such as advanced computer vision models.

3. Building the future

Encouraging the audience to be active contributors to the AI narrative, Terra urged:

It’s not about solving collective problems individually, but about looking to the community to make that decision.

4. Future of work

Soft skills like creativity, originality, and critical thinking will dominate the future, echoing sentiments from the World Economic Forum.

To Terra, many people don’t seem to realize that the decision to substitute humans with machines will not be made by machines, but by other people. However, differentiating yourself by developing soft skills is one way to avoid being replaced as a professional.

Risks of generative AI

1. Opacity and discriminatory biases

Echoing concerns raised by Mirelle C. Bueno, Terra discussed the dangers of opacity in AI models, highlighting the potential for biases and discriminatory practices.

2. Environmental impact, costs, and rights violations

Generative AI carries a heavy ecological footprint. Using the example of GPT-5, Terra questioned the motivations behind massive investments in AI:

What company would invest that amount of money without getting anything in return?

She also emphasized the ethical dilemmas by pointing out that our current society spends 700,000 dollars a day on a machine and only 2 on human beings, referring to the daily cost of GPT’s GPUs and the payment of only 2 USD per day for Kenyan workers who participated in the LLM training.

3. Modulation of interests and tastes

The influence of generative AIs on our preferences and interests can’t be ignored. These tools, with their hidden agendas, pose risks.

More questions than answers

Terra’s lecture concluded with a thought-provoking segment questioning the unchecked proliferation of AI in our lives:

Should we allow these machines to flood our information channels with propaganda and untruths?

Backing concerns raised by tech leaders like Elon Musk, she challenged the unchecked authority of unelected tech moguls:

To her, these decisions cannot be delegated to unelected technology sector leaders whose authority comes solely from their purchasing power.

Is the adoption of artificial intelligence really inevitable at this point? And does it necessarily represent progress to our society? Is anything really worth it in the name of productivity?

We may not yet have the answers to all these questions. But undoubtedly, minds like Kizzy Terra’s can help us find them in the near future.

The MIA meetup 2023 was an initiative sponsored by Nubank. Here, the presence of women is indispensable if we want to ask the big questions, not just in generative AI, but in all matters related to the tech and financial ecosystems.

If you want to contribute to this discussion, join us. Nubank has job openings for women in Data Science!