FedAssistant: Dialog Agents with Two-Side Modeling
Overview
The success of large pre-trained neural generative conversation models benefits from publicly available datasets from various sources. However, some high-quality datasets held by separated companies and organizations are not exploited yet due to security and privacy concerns.
Commercialisation opportunities
Technology licensing agreement / Industry collaboration
Problem addressed
A framework named FedAssistant to address the security and privacy concern by training neural dialog systems in a federated learning setting. The framework can be trained on multiple data owners with no raw data leakage while training and evaluating the models.
Innovation
- Privacy-preserving Dialog systems built on large language models.
- New uses of well-known language models for protecting raw texts.
- A Framework that can avoid raw data transmission during training and inference states.
Key impact
- Framework can be trained on multiple data owners with no raw data leakage while training and evaluating the models.
- Reduce the communication cost between data holders and the parameter server.
- FedAssistant achieves competitive performance as normal GPT-2 on individual data domains.
Application
- Chatbots on edge devices (smart phones)
- Personal assistants
- Customer service chatbots
The Hong Kong University of Science and Technology (HKUST)
The Hong Kong University of Science and Technology (HKUST) (https://hkust.edu.hk/) is a world-class research intensive university that focuses on science, technology and business as well as humanities and social science. HKUST offers an international campus, and a holistic and interdisciplinary pedagogy to nurture well-rounded graduates with global vision, a strong entrepreneurial spirit and innovative thinking. Over 80% of our research work were rated “Internationally excellent” or “world leading” in the Research Assessment Exercise 2020 of Hong Kong’s University Grants Committee. We were ranked 3rd in Times Higher Education’s Young University Rankings 2022, and our graduates were ranked 23rd worldwide and among the best from universities from Asia in Global University Employability Survey 2021.
Enquiry
