The Aria is a set of initiatives to dramatically increase PrestoDB efficiency. Our goal is to achieve a 2-3x decrease in CPU time for Hive queries against tables stored in ORC format. For Aria, We are pursuing improvements in three areas: table scan, repartitioning (exchange, shuffle), and hash join. Nearly 60 percent of our global [...] Read More... The post Aria Presto: Making table scan more efficient appeared first on Facebook Code.
At today’s Systems @Scale conference, we presented Delos, a fundamentally new architecture for building replicated storage systems. Its modular, layered design provides flexibility and simplicity without sacrificing performance or reliability. Delos enables fast time-to-deployment for new storage systems — we deployed an initial version in production within eight months — as well as safe, rapid [...] Read More... The post Delos: Simple, flexible storage for the Facebook control plane appeared first on Facebook Code.
By Kenny Yu Chunqiang (CQ) Tang At today’s Systems @Scale conference, we presented Tupperware, our cluster management system, which orchestrates containers on millions of servers to run nearly all of our services. Since we first deployed Tupperware in 2011, our infrastructure has expanded from one data center to a fleet of 15 geo-distributed data [...] Read More... The post Efficient, reliable cluster management at scale with Tupperware appeared first on Facebook Code.
We run one of the largest microservices deployments in the world, with thousands of services that perform billions of requests per second. Keeping information secure as these services communicate globally is a complex job that requires thoughtful consideration of tradeoffs between performance, security, and operability. We are sharing new details about how we designed our [...] Read More... The post Building Facebook’s service encryption infrastructure appeared first on Facebook Code.
By Pablo Mazzini DHCP is the network management protocol we use for provisioning servers in our production data centers. We use it for bare metal provisioning (to install the operating system) and to assign IP addresses to out-of-band management interfaces. As traffic demands have grown, production engineers have evolved our DHCP infrastructure to make it [...] Read More... The post Extending DHCPLB: The path from load balancer to server appeared first on Facebook Code.
What it is: Pythia is a deep learning framework that supports multitasking in the vision and language domain. Built on our open-source PyTorch framework, the modular, plug-and-play design enables researchers to quickly build, reproduce, and benchmark AI models. Pythia is designed for vision and language tasks, such as answering questions related to visual data and [...] Read More... The post Releasing Pythia for vision and language multimodal AI models appeared first on Facebook Code.
By Michael Cheng As part of Facebook’s long-standing commitment to open standards and the open source community, we have shared data sets and models to help global organizations. We are using our expertise — including a predictive model for mapping electrical grids, disaster maps , and more accurate population density maps — to improve access [...] Read More... The post Facebook joins new Urban Computing Foundation appeared first on Facebook Code.
At F8 2019 day 2, topics included a new method for computer vision, new techniques for self-supervised learning, and a new framework for inclusive AR/VR. Watch the video of the keynote address and summaries of several of the technical sessions. This post will update as session videos are available Keynote address Chief Technology Officer Mike [...] Read More... The post F8 2019 Day 2 keynote and session videos appeared first on Facebook Code.
At F8, Facebook’s annual conference, we open-sourced tools and frameworks that simplify machine learning experimentation and optimization, speed up test execution times, and help solve memory-related performance issues. See the full list below, and click through to get started with our latest releases. Ax Ax is an accessible, general-purpose adaptive experimentation platform for managing, deploying, [...] Read More... The post Open source at F8 2019 appeared first on Facebook Code.
Today, we kicked off F8 2019. Thousands of attendees gathered in San Jose to learn about our latest plans, research, tools, and open source releases. Topics included ensuring Messenger and WhatsApp are the fast, secure, private centers of your social experience; making Communities as central as friends in Facebook; making commerce thrive in Instagram; and [...] Read More... The post F8 2019 Day 1 keynote and session videos appeared first on Facebook Code.
Hash tables provide a fast way to maintain a set of keys or map keys to values, even if the keys are objects, like strings. They are such a ubiquitous tool in computer science that even incremental improvements can have a large impact. The potential for optimization led to a proliferation of hash table implementations [...] Read More... The post Open-sourcing F14 for faster, more memory-efficient hash tables appeared first on Facebook Code.
The session list is now available for F8, Facebook’s annual conference, being held April 30 and May 1, 2019, in San Jose, California. This year, we’ll celebrate the developers, creators, entrepreneurs, and innovators building on our platform and showcase how social technologies can enable the best of what people can do together. Technical sessions will [...] Read More... The post F8 2019 speakers and sessions now available appeared first on Facebook Code.
The time it takes for an interaction to go from the user input event (such as clicking a button or typing in a box) that triggered it to being completely rendered is an important web metric. At Facebook, we measure events at four stages: the moment the operating system gets the input, the moment we [...] Read More... The post Faster input events with Facebook’s first browser API contribution appeared first on Facebook Code.
What the research is: A new approach to object recognition that uses a single neural network to simultaneously recognize distinct foreground objects, such as animals or people (a task called instance segmentation), while also labeling pixels in the image background with classes, such as road, sky, or grass (a task called semantic segmentation). While previous [...] Read More... The post Improving scene understanding through panoptic segmentation appeared first on Facebook Code.
Thousands of engineers write the code to create our apps, which serve billions of people worldwide. This is no trivial task—our services have grown so diverse and complex that the codebase contains millions of lines of code that intersect with a wide variety of different systems, from messaging to image rendering. To simplify and speed [...] Read More... The post Aroma: Using machine learning for code recommendation appeared first on Facebook Code.
This month on our AI blog, we shared some big news about our chief AI scientist, detailed a new approach for multitask Bayesian optimization, and caught up with the researchers in our AI Residency Program. And on Tech@facebook, our new blog for tech enthusiasts, we unpacked how technology is transforming the way we communicate, see [...] Read More... The post Facebook tech news roundup appeared first on Facebook Code.
We recently hosted our first-ever London Facebook Engineering Fair. The invitation-only event gathered together software engineers, product managers, UX researchers, data scientists, academics, and others working in the technology industry. Several London-based teams gave attendees an exclusive behind-the-scenes look at the products and technologies being worked on by our U.K. engineers. Director of Product for [...] Read More... The post Facebook Engineering Fair, London recap appeared first on Facebook Code.
WHAT’S NEW: The first experimental back end for our Glow compiler and runtime project, designed to target Habana’s existing hardware accelerator. This back end is the first to customize for various vendors’ accelerators. WHY IT MATTERS: Glow’s open source framework allows partners to more rapidly design and optimize new silicon products for machine learning (ML). [...] Read More... The post Open-sourcing Habana back end for Glow appeared first on Facebook Code.
WHAT IT IS: A new tool from Facebook AI Research that enables training of multi-relation graph embeddings for very large graphs. PyTorch-BigGraph (PBG) handles graphs with billions of nodes and trillions of edges. Since PBG is written in PyTorch, researchers and engineers can easily swap in their own loss functions, models, and other components. Read [...] Read More... The post PyTorch-BigGraph: Faster embeddings of extremely large graphs appeared first on Facebook Code.
Facebook’s Chief AI Scientist, Yann LeCun, is one of three recipients of this year’s Association for Computing Machinery (ACM) A.M. Turing Award. Sharing the honor is Yoshua Bengio (MILA and University of Montreal) and Geoffrey Hinton (Google, Vector Institute, and University of Toronto). Often referred to as the Nobel Prize of computing, the award recognizes [...] Read More... The post Turing Award presented to Yann LeCun, Geoffrey Hinton, and Yoshua Bengio appeared first on Facebook Code.