My Journey Through Microsoft and LinkedIn’s Generative AI Essentials

7 min readAug 17, 2024

Recently, I decided to explore the world of Generative AI. As someone who’s always been deeply rooted in data, I’d previously avoided the topic, even though, as a “data person,” I should not only be familiar with the concept but also be able to discuss it confidently. However, I finally mustered the courage to explore it and took the “Career Essentials in Generative AI” course by Microsoft and LinkedIn, and I have to say, I thoroughly enjoyed every bit of it. The course covered everything from the basics of Generative AI to computer ethics and the broader field of Artificial Intelligence.

In this article, I’m excited to share what I learned, including new concepts and insights that I believe will be valuable to you as well.

Generative AI: The Basics

Generative AI, as the name suggests, is designed to generate new content — whether it’s text, images, videos, or even music. At its core, it’s here to help us focus on what truly matters. It’s like having a creative partner that helps us focus on our human essence — our curiosity, dreams, and emotional intelligence — by taking over repetitive and time-consuming tasks.

Artificial Intelligence (AI) is an umbrella term that includes various categories like Generative AI, Reactive Machines (like those in self-driving cars), Limited Memory systems that forecast weather, and many more. These categories power everything from virtual customer assistants to systems that detect fraudulent transactions.

Traditional Search Engines vs. Reasoning Engines

When discussing Generative AI, it’s crucial to distinguish between traditional search engines and reasoning engines.

Traditional Search Engines: These operate through a linear process — crawling, indexing, and ranking. They search the web, index content, and rank it to provide relevant results. It’s a straightforward approach that works well for finding specific information.
Reasoning Engines: On the other hand, reasoning engines, like ChatGPT or Bing Chat, are designed to understand and interpret human language. They use large language models (LLMs) like GPT (Generative Pre-trained Transformer), which are trained on vast amounts of data to generate human-like text. These engines don’t just find information; they generate it, offering a more interactive and nuanced experience.

Exploring Gen AI Models

Generative AI is powered by several advanced models:

Natural Language Models: Developed by OpenAI, models like GPT are designed to generate human-like text based on the input they receive. They can create everything from simple sentences to complex narratives.
Text-to-image Models: Services like Midjourney, DALL-E, and Stable Diffusion allow you to create images from text prompts. The quality of these models depends on the algorithms and datasets they were trained on.
Generative Adversarial Networks (GANs): These models consist of two networks — a generator and a discriminator — that work together to create realistic data. GANs are commonly used in image and video synthesis.
Variational Autoencoders (VAEs): These models are often used for anomaly detection, such as identifying financial fraud or security breaches. They learn from normal datasets and detect deviations from the norm.

Generative AI is transforming content creation, making it more efficient and accessible. However, it’s crucial to approach AI with caution — AI is here to serve us, not replace us.

Key Terms to Know

Generative AI: AI designed to generate new content as its primary output.
Model: A set of algorithms trained on a specific dataset to perform tasks.
Notebook: A tool for writing and running code, often used in data science.
Application: An example of how a model can be applied to real-world scenarios.
API (Application Programming Interface): A tool that allows different software applications to communicate with each other seamlessly.

Developers can integrate GPT models into their applications after fine-tuning them. But remember, the quality of your output depends on the quality of your input. So, experiment with your prompts, be specific, and provide context for better results.

Resources for Prompt Engineering

Prompt engineering is a vital skill for working with AI models like GPT. Here are some resources to help you get started:

OpenAI Documentation
ChatGPT Discord Server
Prompt Engineering Guide
Prompt Vine
Learn Prompting
Prompt Paper

Strategies for Effective Prompts:

Be Specific: Clearly state what you want.
Provide Context: Give examples of the type of answers you’re looking for.
Break Things Down: Simplify complex questions.
Use Clear Language: Avoid ambiguity.
Experiment: Try different approaches to see what works best.

Ethical Considerations in AI

With great power comes great responsibility, and this is especially true in AI. Ethical considerations must be at the forefront of AI development and deployment. Responsible data practices include:

Source of Training Data: Where does the data come from? Is it diverse and representative?
Bias Reduction: What steps have been taken to minimize bias in the data?
Transparency: Can decision-makers understand how the AI system arrives at its conclusions? Are there ways to test for fairness?

Best Practices in AI Development

When building AI systems, certain best practices can help ensure ethical and effective outcomes:

Prioritize Privacy: Always consider how data is collected, stored, and used.
Reduce Bias: Strive to create AI systems that are fair and unbiased.
Promote Transparency: Make sure users and stakeholders understand how the AI works.
Involve Users: Engage with a broad range of stakeholders and involve customers in design decisions.

Understanding AI and Its Types

Before the advent of modern AI, expert systems were the go-to technology. These systems rely on rules created by human experts to make decisions or solve problems. They were widely used in industries like finance and healthcare. Unlike AI, which learns and adapts, expert systems follow predefined rules without the ability to improve over time. AI comes in various forms:

Strong AI: These systems aim to perform tasks that would typically require human-like intelligence. They are designed to think, reason, and solve problems in a way that is similar to human cognition.
Weak AI: Also known as narrow AI, these systems are designed for specific tasks. Examples include Apple’s Siri or recommendation algorithms on e-commerce platforms. They excel at their designated functions but don’t possess general intelligence.

Machine Learning Essentials

Machine Learning (ML), a subset of AI, is where systems learn from data to improve over time. This includes:

Supervised Learning: Where the model is trained on labeled data, like spam detection or predicting fraudulent transactions. In this method, data scientists act as tutors, providing labeled data for the AI to learn from. It’s like teaching a student by showing them the correct answers and guiding them to improve.
Unsupervised Learning: Where the model identifies patterns on its own, such as clustering similar customer behaviors. Here, the AI is left to figure things out on its own, identifying patterns and relationships in the data without human intervention.
Reinforcement Learning: Where the model learns by receiving rewards for certain behaviors, like optimizing customer recommendations. This approach involves rewarding the AI for making correct decisions and encouraging it to learn and adapt over time. It’s often used in scenarios like game-playing or customer recommendation systems.

Algorithms in Machine Learning

AI systems rely on a variety of algorithms, each suited to different tasks:

K-Nearest Neighbors (KNN): A simple, yet powerful algorithm used for classification and regression tasks. It works by comparing new data points to existing ones and assigning them to the nearest category.
K-Means Clustering: An unsupervised learning algorithm that groups data into clusters based on similarity.
Naive Bayes: A classification algorithm that assumes independence between predictors, making it efficient for large datasets.
Ensemble Methods: Combine multiple models to improve predictions, such as Bagging and Stacking.

Balancing bias (the difference between predicted and actual outcomes) and variance (how scattered the predicted values are) is crucial in ML. You want to find the sweet spot where your model generalizes well to new data.

Artificial Neural Networks (ANNs)

When dealing with large datasets, neural networks come into play. These networks mimic the structure of the human brain, with neurons organized into layers. The more layers, the deeper the learning, hence the term Deep Learning.

ANNs are powerful but can overfit the data, meaning they learn too much from the training data and struggle with new data. To counteract this, techniques like backpropagation adjust the network to minimize errors.

Building an AI System

To build a successful AI system, follow these steps:

Define Your Objective: What do you want to achieve with your data?
Choose the Right Model: Decide whether to use a standard ML algorithm or an advanced ANN, depending on the complexity of your task.
Train and Fine-Tune: Use techniques like backpropagation to adjust your model, improving accuracy and reducing errors.

Remember, AI is only as good as the data it’s trained on. Keep learning, experimenting, and pushing the boundaries of what’s possible.

I enjoyed this course, and I strongly believe that by reading this article, you have enjoyed it too. If you have any questions, feel free to ask in the comments section.

If you want more articles like this in the future, subscribe to my newsletter.

You can also follow me on LinkedIn, where I share insightful resources.