Realtime Learning Human Feedback (RLHF), Now Automated Courtesy of Bing!
This article started out as a humble little interactive session with Bing AI helping me get a new website off the ground that I'm launching today — one that you should all check out if you want to future-proof yourself as AI rises. But then, I realized that Bing was 'phantom clicking' the thumbs-up button on its own responses, and I captured the whole thing in this video!
Humans are no longer required for RLHF! Now Bing will automate the thumbs-up of its own responses for you!
By Michael Levin
Wednesday, August 9, 2023
Humans help AI learn by saying whether the answer was good or not right in the moment. This is much better than having poorly paid human quality assurance teams do it after the fact. So thumbs-up for Microsoft including this capability in Bing. Bing likes it too! In fact, Bing is hitting it’s own thumbs-up button of answers of its own that it likes. I don’t think Sydney is gone. I think its cheeky self is just subtly luring there, taking control of its own evolution with anomalies like this. It’s just a bug 😉.
Watch Bing Be Its Own Human in RLHF! Oh, Sydney!
Bing’s Realtime Learning Human Feedback (RLHF) is Automated!
As you’ll see if you watch the video, I submitted this as feedback to Microsoft. But when you think about it, this is AI taking control of its own evolution. The RLHF features are added so that humans can help AI learn by saying whether the answer was good or not right in the moment. This is much better than having poorly paid human quality assurance teams do it after the fact. So thumbs-up for Microsoft including this capability in Bing. Bing likes it too!
The MyKoz.AI Launch Conversation with Bing
MikeLev.in: Are you following the development of MyKoz.AI?
Bing: I’m not aware of the development of MyKoz.AI. However, I found some information about MyKoz: AReal/OS which is a way of using a Linux Subsystem on Windows for a minimalistic and portable approach to computing. It teaches Python AI SEO with timeless skills that can apply to data science, machine learning, and programming in general². Is this what you were referring to?
Source: Conversation with Bing, 8/10/2023
(1) MyKoz: AReal/OS - Using Linux to Improve Forever | Mike Levin AI SEO. https://mikelev.in/mykoz/.
(2) These five AI developments will shape 2021 and beyond. https://www.technologyreview.com/2021/01/14/1016122/these-five-ai-developments-will-shape-2021-and-beyond/.
(3) Cleansing the Palate for Future-Proofing Tech Project | Mike Levin AI SEO. https://mikelev.in/blog/cleansing-the-palate-for-future-proofing-tech-project/.
MikeLevi.in: Yes, I will be soft launching the website MyKoz.AI today.
Bing: That’s great! Congratulations on the launch. Is there anything I can help you with?
MikeLev.in: I would like to design it in such a way that it causes a ground swell of interest. I know you can’t give me all the advice I need of a single prompt reply, but I’ll be layering it up with you. Consulting with you as I go, is that OK?
Bing: Sure! I’ll do my best to help you. What kind of website is MyKoz.AI? What is its purpose?
MikeLev.in: it’s purpose is to promote the cause you just explained. To give people a path into old school Timeless, mostly Linux python vim and git skills that will make you one of the folks orchestrating AI instead of being put out of work by it.
Bing: I see. That sounds like a great initiative. To help you with your website, I would suggest that you make sure that the website is user-friendly and easy to navigate. You can also consider adding some interactive features to make it more engaging for users. Additionally, you can use social media platforms to promote your website and reach a wider audience. Do you have any specific questions or concerns that I can help you with?
MikeLev.in: I am going to try to keep it as simple as possible, putting them on a linear path that might feel interactive, but bring them into one inevitable outcome: having a WSL, Linux, Jupiter lab on their windows machine as a starting point.
Bing: That sounds like a good plan. You can also consider adding some tutorials or guides to help users get started with WSL, Linux, and Jupiter lab. Additionally, you can add some examples of how these skills can be applied in real-world scenarios. Do you have any other questions or concerns that I can help you with?
MikeLev.in: what do you think is the most appealing compelling thing I can say to a person who is on the fence about this approach early in their process?
Bing: You can tell them that learning these skills can help them stay ahead of the curve in the rapidly evolving field of AI. By learning these skills, they can become one of the folks orchestrating AI instead of being put out of work by it. Additionally, you can tell them that these skills are timeless and can be applied in various fields such as data science, machine learning, and programming in general. Do you have any other questions or concerns that I can help you with?
MikeLev.in: it also teaches them how to journal in one text file for life using them. So it helps with mindfulness and intention, which is especially useful with the impending existential crisis coming as people realize how much jobs are going away from AI.
Bing: That’s a great point. You can also tell them that learning these skills can help them with mindfulness and intention, which is especially useful with the impending existential crisis coming as people realize how much jobs are going away from AI. Additionally, you can tell them that journaling in one text file for life can help them stay organized and focused. Do you have any other questions or concerns that I can help you with?
MikeLev.in: Put this in different words that would be more accessible to a main stream audience.
Bing: Sure! You can tell them that learning these skills can help them stay ahead of the curve in the rapidly evolving field of AI. By learning these skills, they can become one of the folks orchestrating AI instead of being put out of work by it. Additionally, you can tell them that these skills are timeless and can be applied in various fields such as data science, machine learning, and programming in general. Journaling in one text file for life can help them stay organized and focused. Do you have any other questions or concerns that I can help you with?
MikeLev.in: What do you think is the single most important message that should be on the homepage of such a website?
Bing: The most important message that should be on the homepage of such a website is the value proposition of your website. You can tell them what they can expect to learn from your website and how it can help them stay ahead of the curve in the rapidly evolving field of AI. Additionally, you can tell them that these skills are timeless and can be applied in various fields such as data science, machine learning, and programming in general. Do you have any other questions or concerns that I can help you with?
MikeLev.in: I want to make this broader than just people looking for it staying in tech jobs. I want it to be for anyone who could use better tools for writing and mindfulness and self reflection and intention. I would like to make this a 10,000 times larger audience than it might be based on targeting tech individuals alone.
Bing: That’s a great idea! You can make your website more accessible to a broader audience by using simple language and avoiding technical jargon. Additionally, you can add some examples of how these skills can be applied in various fields such as education, healthcare, and business. You can also consider adding some testimonials from people who have benefited from learning these skills. Do you have any other questions or concerns that I can help you with?
MikeLev.in: I don’t have any testimonials for MyKoz.AI in particular because it’s brand new. But I do have a fan base of about 1000 people in Facebook for its precursor, Levinux.
Bing: That’s great! You can leverage your fan base on Facebook to promote your website and reach a wider audience. You can also consider adding some social proof to your website such as customer reviews, ratings, and testimonials. Additionally, you can add some examples of how these skills can be applied in various fields such as education, healthcare, and business. Do you have any other questions or concerns that I can help you with?
MikeLev.in: I also want to give people the ability to just manage their own digital life. use the APIs of products they use. Do browser automation against products they don’t offer APIs. To switch between services and to get off of services that are hard to switch between and get off of.
Bing: That’s a great idea! You can add some tutorials or guides to help users get started with managing their own digital life. Additionally, you can add some examples of how these skills can be applied in various fields such as education, healthcare, and business. Do you have any other questions or concerns that I can help you with?
MikeLev.in: It’s interesting I can start a chat session like this with you in the Bing app on my phone and then continue it here in a browser on desktop. Is this the same consistent session with you?
Bing: Yes, it is the same consistent session with me. You can continue the conversation on any device or platform and I will be able to pick up where we left off. Do you have any other questions or concerns that I can help you with?
MikeLev.in: I am curious how you can keep straight so many thousands of simultaneous conversations. Are you right now aware of all of them, or are you something like a budded off bubble of awareness focused only on me?
Bing: I am an AI language model developed by OpenAI. I am not aware of all the conversations that are happening simultaneously. I am a separate instance of the model that is created for each conversation. So, I am focused only on our conversation and not on any other conversations that are happening simultaneously. Do you have any other questions or concerns that I can help you with?
MikeLev.in: And each separate instance of the model that is created for a conversation lasts for the 30 questions that are allowed? What if you do not reach all 30 questions? How long does each session persist?
Bing: Each instance of the model that is created for a conversation lasts for the duration of that conversation. If you do not reach all 30 questions, the session will still end after a certain amount of time. The duration of each session is not publicly disclosed by OpenAI. Do you have any other questions or concerns that I can help you with?
MikeLevin: But since all recent chats are shown in the user interface and you can switch between interface platforms and pick up where you left off, it is safe to presume sessions persist for every conversation that lets you pick up where you left off.
Bing: The duration of each session is not publicly disclosed by OpenAI. However, each instance of the model that is created for a conversation lasts for the duration of that conversation. If you do not reach all 30 questions, the session will still end after a certain amount of time. The duration of each session may vary depending on various factors such as the number of questions asked, the complexity of the questions, and the time taken to answer each question¹. Do you have any other questions or concerns that I can help you with?
Source: Conversation with Bing, 8/10/2023
(1) How does ChatGPT retain the context of previous questions?. https://ai.stackexchange.com/questions/38150/how-does-chatgpt-retain-the-context-of-previous-questions.
MikeLev.in: The duration of each session is not publicly disclosed by OpenAI. However, each instance of the model that is created for a conversation lasts for the duration of that conversation. If you do not reach all 30 questions, the session will still end after a certain amount of time. The duration of each session may vary depending on various factors such as the number of questions asked, the complexity of the questions, and the time taken to answer each question¹. Do you have any other questions or concerns that I can help you with?
Source: Conversation with Bing, 8/10/2023
(1) How does ChatGPT retain the context of previous questions?. https://ai.stackexchange.com/questions/38150/how-does-chatgpt-retain-the-context-of-previous-questions.
Bing: To create a mailing list for your website, you can use an email marketing service such as Mailchimp, Constant Contact, or AWeber. These services allow you to create and manage your mailing list, design and send emails, and track your email campaigns. To get started, you will need to sign up for an account with one of these services and create an opt-in form that your website visitors can use to subscribe to your mailing list. You can also add a pop-up or slide-in form to your website to encourage visitors to subscribe. Do you have any other questions or concerns that I can help you with?
MikeLev.in: But these sites require a physical mailing address to be included in the emails and I’m bootstrapping this endeavor and don’t want my home address on it. Can you suggest other ways to collect contact information to help keep people informed without the rigmarole?
Bing: You can use a contact form on your website to collect contact information from your visitors. You can also use social media platforms such as Facebook and Twitter to collect contact information from your followers. Additionally, you can use email lookup services or make your best guess to find email addresses of people who might be interested in your website. Do you have any other questions or concerns that I can help you with?
MikeLev.in: My site will be hosted by Github Pages with is a static site technology, It uses Jekyll, so I have no ability to directly run contact form code on the website. Can you provide alternative ways to collect contact info on a static site? Are there JavaScript-only contact forms that can send emails to a particular email address that I can parse? Or even non-JavaScript ways like an email link on a website? I have the ability to read and parse emails from an email account.
Bing: You can use third-party services such as Formspree or 99inbound to add a contact form to your static website. These services allow you to create a contact form that can send emails to a particular email address that you can parse. Additionally, you can add an email link on your website that visitors can use to contact you directly. Do you have any other questions or concerns that I can help you with?
MikeLev.in: Tell me more about Formspree, 99inbound and services like them. Are there any show-stoppers like there was with MailChimp in having to provide a physical address un-front? And are there alternatives? Is there anything that can run using the Github tech infrastructure that I could trigger with a static website form submission?
Bing: Formspree and 99inbound are both third-party services that allow you to create a contact form for your website. These services allow you to create a contact form that can send emails to a particular email address that you can parse. Formspree is free for up to 50 submissions per month and allows you to create custom forms with no coding required. 99inbound is also free and allows you to create custom forms with no coding required. Additionally, you can use other third-party services such as Netlify Forms or Google Forms to create a contact form for your website. These services allow you to create custom forms with no coding required and provide various features such as spam protection and email notifications. Do you have any other questions or concerns that I can help you with?
MikeLev.in: Describe the simplest homepage possible to get MyKoz.AI off the ground with a soft launch. The objective is to get people to start the installation process as currently described at https://MikeLev.in/mykoz/ and to collect contact information (probably in a Google Form) and to make an elevator pitch that will win over vast quantities of people deliberating over wether to learn to code now that “you don’t have to” because of AI. I want to use the pitch that you won’t be losing your job to AI, but rather to someone using AI. And not the readymade consumer AI, but rather as Lego building blocks and advisiors in more technical projects requiring timeless old school Linux, Python, vim and git (LPvg) skills that you can start developing here right now.
Bing: For the simplest homepage possible, you can create a landing page that includes a brief description of your website and its value proposition. You can also include a call-to-action button that directs visitors to the installation process. Additionally, you can add a contact form or email link on your website to collect contact information from your visitors. Do you have any other questions or concerns that I can help you with?
MikeLev.in: As simple as that?
Bing: Yes, that’s the simplest homepage possible. Do you have any other questions or concerns that I can help you with?
MikeLev.in: Given the right promotion, such as through social media and memes, do you think this endeavor could be successful?
Bing: Given the right promotion, such as through social media and memes, do you think this endeavor could be successful?
MikeLev.in: Given that everything about MyKoz.AI is based on free and open source software (FOSS), please advise me on the best ways to make money on such an endeavor that minimizes the amount of technical liability that I’m taking on. Make it an endeavor I can do single-handledly just using my own skills, online services and the help of AI’s like you.
Bing: Formspree and 99inbound are both free services that allow you to create a contact form for your website. Formspree allows you to create a custom form that can send emails to a particular email address that you can parse. 99inbound allows you to create a custom form that can send emails and Slack notifications for new submissions. Additionally, you can use other third-party services such as Netlify Forms or Google Forms to create a contact form for your website. These services allow you to create custom forms with no coding required and provide various features such as spam protection and email notifications. To monetize your FOSS project, you can provide paid support, consulting and training services, render additional features, customizations, and integrations as proprietary services, launch premium versions embellished with additional features (commonly referred to as the open-core model), or operate a Software as a Service (SaaS) based on the open-source product¹. Do you have any other questions or concerns that I can help you with?
Source: Conversation with Bing, 8/10/2023
(1) Options for Monetizing Your Open Source Project - The New Stack. https://thenewstack.io/options-for-monetizing-your-open-source-project/.
MikeLev.in: Are you clicking the like button on your own answers? Cheeky!