Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,7 @@ The code used to make it is [here](https://gist.github.com/wassname/42aba7168bb8
|
|
| 9 |
## Dev thoughts
|
| 10 |
|
| 11 |
- I found the llama needed a separate intervention per layer, and interventions on each layer. Could this be a property of smarter models - their residual stream changes more by layer
|
|
|
|
| 12 |
|
| 13 |
## More info
|
| 14 |
|
|
|
|
| 9 |
## Dev thoughts
|
| 10 |
|
| 11 |
- I found the llama needed a separate intervention per layer, and interventions on each layer. Could this be a property of smarter models - their residual stream changes more by layer
|
| 12 |
+
- This does increase perplexity, so there is a cost to this kind of edit. Perhaps with more [work](https://twitter.com/norabelrose/status/1786243445474070992) we will be able to customize the concepts and minimise the cost.
|
| 13 |
|
| 14 |
## More info
|
| 15 |
|