You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At some point, using initial_session_file in AgentEvaluator.evaluate worked; but now the session can only be initialized from the evalset file itself. Having to edit the evalset is cumbersome compared to maintaining a CLI option which could help when injecting secrets and tokens as well, vs polluting a tracked evalset file and having to develop a custom injection into session_input.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
At some point, using
initial_session_file
inAgentEvaluator.evaluate
worked; but now the session can only be initialized from the evalset file itself. Having to edit the evalset is cumbersome compared to maintaining a CLI option which could help when injecting secrets and tokens as well, vs polluting a tracked evalset file and having to develop a custom injection into session_input.Beta Was this translation helpful? Give feedback.
All reactions