From left: Principal Researcher Thomas Ball, Research Interns Teddy Seyed and James Devine, and Principal Research Software Development Engineer Peli de Halleux helped designers of the Brooklyn Public Library’s BKLYN Fashion Academy enhance their designs with custom-made plug-and-play microcontroller technology. Photo credit: Kristen Blush
In his work in visual merchandising, designer Kenroy Tyrell has used LED lights in displays for trade shows and showrooms. But on this particular day, he found himself using the technology in a way he never anticipated—as part of a garment he had created.
Tyrell was one of 20 participants in the Brooklyn Public Library’s BKLYN Fashion Academy, a program geared toward exposing designers to the entrepreneurial side of the industry as they design collections. According to the library, it’s the only library in the country with a fashion school, and in its second year, the program snagged a unique partner in Microsoft Research. Researchers teamed up with participants—most of whom had little to no technical experience prior to the program—to enhance the storytelling power of their work using custom-made plug-and-play microcontroller technology.
Microsoft Research Blog
Microsoft Research Forum Episode 3: Globally inclusive and equitable AI, new use cases for AI, and more
In the latest episode of Microsoft Research Forum, researchers explored the importance of globally inclusive and equitable AI, shared updates on AutoGen and MatterGen, presented novel use cases for AI, including industrial applications and the potential of multimodal models to improve assistive technologies.
With the start of the academy’s end-of-program fashion show quickly approaching, Tyrell was shaking the legs of a pair of floor-length trousers in an attempt to mimic its functionality. Each of his model’s steps was supposed to activate a motion sensor that would initiate a pulsating light on the tail of the garment. During this test run, though, the LEDs were only lighting sometimes and required a harder shake than he thought his model would generate walking the runway. The sensitivity of the sensor would have to be increased.
Other designers were troubleshooting challenges of their own, right down to the minutes before their respective models were set to walk. The technical look in Tammy Vaughn’s collection incorporated glass-covered motors designed to rotate on the shoulders of her garment, but the motors were heavier than expected, requiring her to reinforce the shoulders with whatever she and those helping her could find, mainly newspaper and plastic bags. Omega Dale was facing motor issues of a different kind. The pair she had included in her garment to slowly lift and drop the front of a skirt seemed to be working at different speeds; after a couple of rounds, they were out of sync, and ultimately one side snapped.
By the time their models took to the runway, Tyrell’s LEDs were working as planned; Vaughn’s motors were more or less rotating, but wound up not reflecting light exactly as envisioned; and Dale had made the executive decision to disengage the tech in her garment that had still been functioning.
But perfection wasn’t the point as it rarely is when it comes to driving the cutting edge. The runway show was important for another reason: to gauge how usable the technology was and how the researchers could advance and scale it. The tech was the first application of a larger project called Alava, which aims to develop microcontroller-based systems easy enough to build and code that those with a limited computer science background could create with them on their own.
“The best way to test your platform is to actually get it into the hands of people, and that was one of the things I’ve always done with a lot of my research,” says Microsoft Research Intern and University of Calgary grad student Teddy Seyed, who facilitated the collaboration. “I like to do work out in the real world and not just keep it in a lab.”
And it doesn’t get more real than the runway, as Microsoft Fusionist Asta Roseway can attest to. Having attended The New School’s Parsons School of Design, which offers a fashion program, she’s witnessed the intensity and, at times, tears of runway shows.
“It’s the perfect test bed to see if people could quickly pull something together or be really creative in a pinch: Where are their limitations? How far can we stretch this technology? It’s the perfect storm for that because there’s no shortage of creativity there,” she says. Her group’s Smart Tattoos—capacitive material that responds to touch and can be laser cut and applied to skin and fabric—was incorporated into some of the garments.
Designer Kenroy Tyrell incorporated LED lights that pulsated with each of his model’s steps into one of his looks, a cropped horned corset and floor-length trousers, for the BKLYN Fashion Academy runway show. The lights were activated by motion sensors in the model’s pant legs. Photo credits: Kristen Blush
When fashion and tech collide
For Microsoft Research, the fashion show on May 3 at the Brooklyn Public Library’s Central Library marked a convergence of journeys.
After years of developing tools for professional developers, Principal Researcher Thomas Ball and colleagues in the Research in Software Engineering group turned their focus to making coding accessible to the non-professional, specifically kids and teens who might not otherwise be introduced to or know they have an interest in computer science. The team’s work led to Microsoft MakeCode, an open-source web-based platform for microcontroller programming consisting of two editors. The first is Google’s Blockly editor for drag-and-drop programming through a partnership with the Google Blockly team; the second is a TypeScript editor based on the one used in Visual Studio Code for more advanced learning and functionality. It also offers a hardware simulator for running and testing code, making it easier for those without access to hardware to start coding.
At about the same time MakeCode was picking up speed, Seyed was being criticized for a smart watch he’d created. A very vocal critic said it was too clunky, not very aesthetically pleasing. Seyed, who takes pride in making things that both look good and are functional, turned to fashion, and what started out as a mission to make better smart watches morphed into doing wearables on the fashion side.
“I wanted the tech to be in the clothing, but I’m not the one who’s going to be able to do the clothing design,” he recalls thinking at the time. “There’s other people who will be able to do that. But if I can give them tech that lets them do that, then I can have what I want—theoretically.”
So Seyed started teaming up with fashion designers and prototyping with such tools as Arduino. He quickly found the tech was too complicated for designers and had several barriers of entry. The technical aspect was distracting from what the technology could do and how designers could elevate their work with it. In search of an easier way, Seyed discovered MakeCode and pitched adapting it for the world of fashion. The MakeCode team—which includes Ball, Principal Research Software Development Engineers Peli de Halleux and Michal Moskal, and Principal Researcher Steve Hodges—was immediately on board. But it would be another year before Seyed joined the team, and in that time, another intern, Lancaster University grad student James Devine, would help develop a networking system that opened the door to using MakeCode in ways not previously considered.
For the fashion application, the plug-and-play microcontroller technology included two core hardware components, both custom-made with SAMD21 microcontrollers: the “brain” and the “beads.” Each of the color-coded beads, or sensors, responded to a different stimulus. The red bead (above, right) is a light sensor bead. Photo credit: Kristen Blush
JACDAC: From multiplayer gaming to a web of microcontrollers
In his second internship with Microsoft Research, Devine was expecting to write a wireless protocol for the BBC micro:bit, but was pulled in to work on MakeCode’s Arcade, an editor for programming single-player video games. The MakeCode team wanted to give Arcade a multiplayer option.
The challenge was existing communication protocols I2C and SPI followed a host-peripheral communications paradigm: One device—the host—is responsible for the operation of all other devices, or peripherals. Without writing customized device drivers, the host wouldn’t be able to distinguish between additional peripherals with the same registers and addresses, preventing the system from functioning. Moreover, the static nature of these protocols made on-the-fly replacement and addition of hardware difficult.
Such limitations were counter to the MakeCode mission of coding for all, and so Devine developed JACDAC—Joint Asynchronous Communications; Device Agnostic Control—a plug-and-play broadcast protocol for microcontroller-based devices. Using dynamic addressing and a layer of abstraction, the protocol standardizes the interface between peripherals, accommodating more than one of the same devices and enabling host-to-host communication for use cases such as multiplayer gaming. The breakthrough inspired what eventually made it down the runway—not just single devices, but a network of them.
“Our innovation is making it really easy to plug these boards together and program them,” says Ball.
JACDAC, which wound up being upgraded for the fashion show, is written in TypeScript and C++, making it usable in both web environments and on microcontrollers. With single-wire capability for transmission and reception, JACDAC allows for the connection of devices via audio cable, a relatively intuitive piece of equipment familiar to the non-tech population and practically made to withstand the fast-paced scenario of fashion design.
“With an audio connector, you get this very distinct, very easy ability to plug things together,” says Devine. “They’re quite sturdy, so if you pull on them, they sort of stick together, and you can only connect them one way like with USB-C.”
Project Alava: Bringing together networking, hardware, and software
One of the unique aspects of Project Alava—and its application for fashion, which the team has dubbed Project Brookdale—is its combination of networking, software, and hardware. On the software side, the MakeCode team leverages the MakeCode Maker editor, an experimentation platform that—like MakeCode—allows drag-and-drop and text-based programming. Hardware for the runway show included two core components, both custom-made with SAMD21 microcontrollers: the “brain” and the “beads.” Each of the color-coded beads, or sensors, responded to a different stimulus. There was a light sensor bead, a color sensor bead, and a motion sensor bead, among others. The team determined what sensing capabilities to build based on the effects designers were hoping to achieve.
“They came up with their concepts, and then it was, ‘What’s the story you’re trying to tell, and how do we make that a reality with the tech?’” says Seyed.
The experience of collaborating with Microsoft researchers had designer Omega Dale thinking about how tech could be incorporated into fashion in utilitarian ways, such as a backpack or windbreaker outfitted with turn signals activated by sensors in the sleeve for cyclists. Photo credit: Kristen Blush
In putting the hardware together—which was made possible with the help of Principal Mechanical Engineer Patrick Therien and his staff at the Hardware Lab—the team was mindful of the designers’ logistical needs. For example, they standardized the size of the beads so no matter what the designers decided to do technically, they would know exactly how much space to allocate in the garment. When it came time to choose a power source, the researchers selected batteries that were on the thinner and lighter side so they wouldn’t drag down the garment.
To create systems those without computer science backgrounds will actually want to use—and be able to sans researchers—requires researchers fit the technology into the worlds where people do their work, whether that be on the runway, in the classroom, or at home.
“You kind of meet them where they are,” says de Halleux.
And the researchers did. After months of walking the designers through the design and technical aspects via phone, email, and videoconferencing from the Microsoft campus in Redmond, Washington, Seyed joined the designers in the workrooms at the library in the two and a half weeks leading up to the runway show. Ball, de Halleux, and Devine were not far behind. And the two teams worked alongside each other—the designers sewing and fitting their garments, the researchers coding and helping to integrate the tech.
Designer Tammy Vaughn, who had incorporated glass-covered motors into her garment, enjoyed using the technology. “I would attempt to incorporate it in something else that I do because it really gives off a cool effect,” she says. Photo credit: Kristen Blush
Designer Tammy Vaughn, who had incorporated the glass-covered motors, enjoyed the collaboration, noting the researchers supported and encouraged them all the way down to the day of the show. As for the tech itself, she’d love to do it again.
“I would attempt to incorporate it in something else that I do because it really gives off a cool effect,” she says.
Kenroy Tyrell and Omega Dale were equally receptive to the idea of continuing to use tech and several days after the show were considering what that might look like moving forward, both tossing around some ideas for more practical applications. Dale talked about a backpack or windbreaker outfitted with turn signals activated by sensors in the sleeve for cyclists. Tyrell mentioned sensors in shoes that could activate the opening of doors in one’s home or car.
“I didn’t think of technology in the sense of sensors in clothes; the sensor thing made me kind of think of other ways to incorporate sensors,” he says.
Such awareness was as much a part of the MakeCode team’s vision as the tech itself, which is still a prototype, and the collaboration was important in showcasing what’s possible not only in fashion, but also in other domains.
“Fashion is just the first step for us to show that the technology is viable—to evaluate it with an audience that is not familiar with microcontrollers—but our aspirations are to work with many different domains,” says Ball.
To experiment with the different components behind Project Alava and Project Brookdale, check out MakeCode Maker and MakeCode. For more information, email the MakeCode team.
Hardware and devices 
Human-computer interaction