Teachable Machine Extension doesn't work in Main branch #368
Comments
|
Thanks for your report, @ctphillips and for trying out both endpoints -- incredibly helpful! For the microbit, are you using Scratch Link? If so, then that issue with the "create" branch is at least expected (if not ideal). There were some changes made to Scratch Link made by the Scratch team after our deployment of the "create" branch which caused some breaking changes -- we also attempt to preserve the "create" branch with minimal changes, which is why it remains broken. However, those issues with the teachable machine extension on the main branch is something we definitely want to fix! @mayarajan3 could you take a peek at this? My suspicion is that things have only broken for audio models (I at least can get image models working consistently once turning video on, let me know if you see any different @ctphillips ). I also suspect when updating this extension, we opted to not have the microphone on by default (as it at least seems in the |
|
Yes, definitely using Scratch Link, so at least it is no surprise to y'all
that connecting to the Micro:bit via Scratch link isn't working in the
create branch.
Definitely, using an Audio Project on the Teachable Machine. I made sure
the microphone was connected and unmuted on the page. I tried both Chrome,
Edge and on my mac I tried Safari. All with the same issue. It is important
to note when the audio model loads (paste the link into "use model" block
and click on it) in the create branch if you are showing the model
prediction variable on the Scratch stage, you immediately see the "noise
background" value get set in the variable. That same sequence does not
produce the expected noise background value in the model prediction
variable (no value is set), which should happen even if the microphone is
muted, since it is the default output of the model.
One other small observation. When you click on the Add Extension icon the
Teachable Machine extension the choices look different between the main
endpoint and create endpoint. The graphic for each are a bit different, and
even the descriptions of the extension are different (of course they are
both titled Teachable Machine, and the extensions themselves show the same
block choices) but the Teachable Machine extension description in the main
endpoint says "Use your Teachable Machine models in your Scratch project",
while for the create endpoint the description says "Use Google Teachable
Machine models in your Scratch project". It makes me think they are loading
different versions of the Teachable Machine extension, but I don't know how
to verify that.
Look forward to hearing what y'all figure out. I would love to use my audio
Teachable Machine with my micro:bit in Scratch. Thanks for all the work
that has been put into thus far!
…-Craig
On Mon, Jun 24, 2024 at 1:15 PM Parker Malachowsky ***@***.***> wrote:
Thanks for your report, @ctphillips <https://github.com/ctphillips> and
for trying out both endpoints -- incredibly helpful!
For the microbit, are you using Scratch Link? If so, then that issue with
the "create" branch is at least expected (if not ideal). There were some
changes made to Scratch Link after the deployment of the "create" branch
which caused some breaking changes -- we also attempt to preserve the
"create" branch with minimal changes, which is why it remains broken.
However, those issues with the teachable machine extension on the main
branch is something we definitely want to fix!
@mayarajan3 <https://github.com/mayarajan3> could you take a peek at
this? My suspicion is that things have only broken for audio models (I at
least can get image models working consistently once turning video on, let
me know if you see any different @ctphillips
<https://github.com/ctphillips> ).
I also suspect when updating this extension, we opted to not have the
microphone on by default (as it at least seems in the main branch we no
longer turn the video on by default, but we do in the create branch).
This is definitely an issue since there is no block to control whether or
not the microphone is recording.
—
Reply to this email directly, view it on GitHub
<https://github.com/mitmedialab/prg-extension-boilerplate/issues/368#issuecomment-2187143826>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV6AFMLS5LK7VTOP45UPJ3ZJBO27AVCNFSM6AAAAABJ2EPJRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGE2DGOBSGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hey @ctphillips -- all sounds good, and you are absolutely correct -- the teachable machine extension in the Thanks to @mayarajan3 's quick work (you rock, @mayarajan3 ), we deployed the likely fix: https://playground.raise.mit.edu/teachable-machine-audio-fix/ Want to make sure it works for your project and then we'll deploy the fix to |
|
Thank you @mayarajan3 for putting together a fix! I verified that the fix
endpoint does allow me to use a teachable machine audio model and connect
to the micro:bit via Scratch Link.
One of the things I was trying to accomplish in this project was to use
audio commands (via the teachable machine model) to have the micro:bit
display icons on its LED array (which I can do using the micro:bit
extension) and drive a couple of servos (I am using motor:bit board from
Elecfreaks) which I cannot do, as there a no pin/servo blocks available in
the micro:bit Scratch extension that I see. Is this something that could be
added to the PRG project? I am looking at other ways to connect Scratch and
micro:bit and get servo control, but I haven't seen anything slightly
straightforward yet.
Thanks!
…-Craig
On Wed, Jun 26, 2024 at 11:48 AM Parker Malachowsky < ***@***.***> wrote:
Hey @ctphillips <https://github.com/ctphillips> -- all sounds good, and
you are absolutely correct -- the teachable machine extension in the main
endpoint is a later version than the one in create.
Thanks to @mayarajan3 <https://github.com/mayarajan3> 's quick work (you
rock, @mayarajan3 <https://github.com/mayarajan3> ), we deployed the
likely fix: https://playground.raise.mit.edu/teachable-machine-audio-fix/
Want to make sure it works for your project and then we'll deploy the fix
to main?
—
Reply to this email directly, view it on GitHub
<https://github.com/mitmedialab/prg-extension-boilerplate/issues/368#issuecomment-2192190482>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV6AFNJ26YNRNYDNEMVUW3ZJLWE3AVCNFSM6AAAAABJ2EPJRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSGE4TANBYGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Trying to use Extension for Teachable Machine with Micro:bit. Oddly, the Micro:bit extension (along with Scratch Link 2.0.2) works fine in the main endpoint (https://playground.raise.mit.edu/main/) but does not work (will not recognize any micro:bits) in the create endpoint (https://playground.raise.mit.edu/create/) same PC, same micro:bit. And the Teachable extension that is provided with the main endpoint, does not provide model outputs (the model loads, but when you provide say the audio that the model expects, the output doesn't update, even the model prediction variable shows no value, not even the background noise output, the variable simply shows nothing), however, the Teachable extension that is provided with the create endpoint works as expected. So I cannot connect Teachable Machine extension to the a micro:bit because in main, I can connect to the micro:bit but the Teachable model does not function, and in create, the Teachable extension works, but I cannot connect to a micro:bit. Not sure where to go from here, so reaching out to see if someone can assist.
The text was updated successfully, but these errors were encountered: