hate-speech-and-offensive-language VS toxicity

I've used NRCLex to perform sentiment analysis on some Twitter data. I have hate speech classifier code (https://github.com/t-davidson/hate-speech-and-offensive-language/blob/master/classifier/final_classifier.ipynb) I want to pass the dataset through, but before I can I need to have a "class" column for the model. For those not familiar, NRCLex returns scores for 10 emotions: anticipation, joy, anger, fear, surprise, disgust, positive, negative, sadness and trust. The table looks like this (letters denoting emotions):

Some of this exists, and both Quora and Facebook (among others) use it extensively. Both hate speech and porn are good targets for machine learning. It needs supervision, but it can take a lot of load off human moderators.

Open source implementations exist, e.g.:

https://github.com/t-davidson/hate-speech-and-offensive-lang...

I suspect more message board will want to start applying these sooner rather than later. Most have already figured out that they need anti-spam tools, rather than it coming as a surprise when they roll things out and it fills up with bots. The technology is similar.

You mention being able to share that information across boards, and I don't know of any widespread implementation of that. You can, at least, let somebody else handle your authentication, which slightly slows their ability to create new accounts when you blacklist one. I'd like to see those sites distinguish "aged" accounts, so that it at least takes some effort or cost to use a new account.

Hmm, I don't want to talk specifics about my experience, but maybe check out some of the case studies on Surge's website - https://www.surgehq.ai/ (about halfway down the page).

> Do they ask PhD to explain root of negative 1 and why is it complex?

This isn't necessarily impossible, but I would consider it to be infeasible with existing labelling workforces. Of course if you really needed a dataset like this and you were sufficiently resourced and willing to spend, you could maybe make it work (I would question whether you really needed PhDs though, that might be hard to swing at any price point).

But the core idea behind your question is correct - this is what a dataset might look like and hiring/contracting appropriately-skilled labelers/generators is how you would go about getting it. Depending on the need, it can be quite a bit more complex too - if you needed self-driving car driving behavior data maybe you build a simulator and hire people to drive in the simulator and use that as training data (made up and probably crap example, but it illustrates the possibilities).

Some people think that labelling workforces are all low skill and there is a lot of good things low skill workforces can do well (visual stuff, basic language and emotion tasks), but you might be surprised at the ability to get skilled labelers. There are lots of smart/educated people around the world and there is ridiculous amounts of money flowing into this space.

I'm a PM at a human data company (https://www.surgehq.ai) that helps the large language model companies ensure their models are safe (we're the “clever prompt engineers” who helped Redwood assess their model performance).

We actually just published a blog today that includes our perspective on building “AI red teams” and best practices for AI alignment/safety: https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-tra...

I'd love to chat. Want to reach out to the email in my profile? I'm the founder of a much higher-quality data startup (https://www.surgehq.ai), and previously built the human computation platforms at a couple FAANGs.

We work with a lot of the top AI/NLP companies and research labs, and do both the "typical" data labeling work (sentiment analysis, text categorization, etc), but also a lot more advanced stuff (e.g., training coding assistants, evaluating the new wave of large language models, adversarial labeling, etc -- so not just distinguishing cats and dogs, but rather making full use of the power of the human mind!).

> Rather than operating under a strict definition of toxicity, we asked our team to identify comments that they personally found toxic.

[0]: https://github.com/surge-ai/toxicity

Love language? So do we, and our mission is to infuse AI with that same love. At Surge, we're building the human infrastructure to power NLP — from detecting hate speech, to parsing complex documents, to injecting human values into the next wave of language models. Our first product is a platform that helps ML teams create amazing, human-powered datasets to train AI in the richness of language. We're a team of former Google, Facebook, and Airbnb engineering leads, and we work with top companies at the forefront of machine learning. Our tech stack is Ruby on Rails, React, and Python. We’re rapidly growing, and we're looking for full-stack engineers to join the team and develop our product. To apply, please email [email protected] with a resume and 2-3 sentences describing your interest in Surge. We love personal projects and writings too!

More information: https://www.surgehq.ai/about#careers

A blog post explaining the problems we are working to solve: https://www.surgehq.ai/blog/the-ai-bottleneck-high-quality-h...