Byung-Gon Chun is CEO and founder of FriendliAI, which helps businesses innovate with their generative AI models. He is also a professor at Seoul National University.
Despite the potential of generative AI, its use involves considerable barriers for startup developers. Developers are thrown into the task of training and serving their models, setting up necessary resources and environments, and handling faults and performance problems. While all these steps are indispensable, they are also incredibly time-consuming and costly. This is where FriendliAI comes in – it helps streamline the process by allowing startups to build and serve their generative AI model using the cloud to meet their requirements quickly and efficiently.
Building FriendliAI
With OpenAI’s release of GPT3 in 2020, I immediately saw an immense opportunity to use my experience in highly scalable machine learning systems. With a decade of experience in distributed computing technologies, both as a professor of Computer Science and Engineering at Seoul National University and as a Principal Scientist at Microsoft, I was excited to explore new opportunities in this realm. Furthermore, as the active commercialization of generative AI began flourishing in different industries, it became evident that advancements in distributed AI computing technologies would also be necessary.
Two years ago, I founded FriendliAI to provide startups with easy access to innovative generative AI. We first helped realize this mission by introducing PeriFlow – a solution to train and serve generative AI models effortlessly with remarkable training and inference speeds and minimal costs. Our ambition has taken us on an incredible journey. We have a unique position within the market as a provider specialized in generative AI training and serving. As a result, we are already partnering with multiple clients to develop and serve cutting-edge generative AI applications powered by PeriFlow.
Fast, Convenient, and Cost-Efficient Ways to Serve and Train Generative AI Models
PeriFlow is an automated platform that handles the ML pipeline from training to serving AI models. It has successfully lowered technical barriers and high costs associated with generative AI services as it not only enables the training and inference process to run smoothly and cost-efficiently but also allows for automation in areas such as resolving faults and performance problems – which would usually require tedious manual monitoring. With PeriFlow, companies, especially , can rest assured that their AI workflow will run at peak efficiency on the cloud; their generative AI inference becomes up to tens of times faster than the conventional inference .
FriendliAI is a leading example of how Azure can assist startups in their growth and success. Through Azure’s AI infrastructure and the PeriFlow platform, users can effortlessly transition cloud services without sacrificing performance or reliability when scaling generative AI models from the laptop to the cloud. Countless organizations rely on the services that Azure offers – proving its immense value within the Microsoft for Startups Founders Hub.
On top of that, PeriFlow helps users find the most optimal combination of generative AI models and cloud resources in a much simpler way allowing users to scale their models in a hassle-free manner. With these features, startups can confidently leverage PeriFlow’s training and serving capabilities while taking advantage of significantly higher request throughput at minimum costs.
Enabling Human-like Communication at Minimal Costs
One customer success story includes ScatterLab’s Luda Lee 2.0, an Open-domain Conversational AI service that has made a major splash in the technology industry, rising to the top spot of both Korea’s Google Play and Apple App Store. What makes Luda Lee 2.0 extraordinary is its impressive contextual prowess—this chatbot can recognize nuanced semantic differences and respond naturally to buzzwords or jokes, prompting even philosophical replies like “Everyone is precious regardless of race.” With the ScatterLab team’s innovation and the PeriFlow platform’s features, the impressive chatbot now services a 17X larger model with remarkable speeds and minimal costs. ScatterLab has set a new gold standard for Open-domain Conversational AI services in Korea.
Choosing the Perfect Cloud Environment for an AI Platform
Cloud computing is driving innovation and transforming the industry. With the PeriFlow Platform leading generative AI innovation through Azure cloud, users can retain performance, connectivity, and flexibility guaranteed on a scalable basis. These are some of the Azure services that PeriFlow has leveraged to offer secure, reliable results:
- Azure Virtual Machines (VMs): Azure supports GPU-enabled virtual machines for machine learning training and serving. ND and NC VMs are scalable for multi-GPU training and serving with high-speed interconnects using NVLink and Infiniband.
- Azure Blob Storage: Azure Blob Storage allows uploading a custom dataset. PeriFlow reads the dataset, trains the model, and creates checkpoints, which periodically send to the storage for future use in the PeriFlow serving service.
- Azure Kubernetes Service (AKS): AKS integrates with PeriFlow for deploying serving engines for elastic provisioning, cost management, and security.
With Azure’s best practices in operational excellence, cost optimization, and sustainability – Azure gives PeriFlow and its users the basis necessary for a streamlined experience configuring production environments while maintaining efficiency and business continuity.
For startups getting started with AI
The new wave of AI services can be intimidating, especially for early-stage startups. The key to our initial success were to stay focused on developing the best software for generative AI, start small, and work with early generative AI customers to deliver and improve what we provide. This customer-centric viewpoint has been critical for us to move nimbly and stay focused. I am delighted to be part of this incredible journey and endeavor.
For more resources for launching your startup and leveraging AI services like Azure AI or OpenAI, sign up today for Microsoft for Startups Founders Hub.
