Particle Swarm Optimization
import random, math
def pso(objective, bounds, n_particles=30, iterations=100):
dim = len(bounds)
particles = [[random.uniform(b[0], b[1]) for b in bounds] for _ in range(n_particles)]
velocity = [[0]*dim for _ in range(n_particles)]
pbest = [p[:] for p in particles]
pbest_val = [objective(p) for p in particles]
gbest = min(pbest, key=lambda p: objective(p))
gbest_val = objective(gbest)
w, c1, c2 = 0.7, 1.5, 1.5
for _ in range(iterations):
for i in range(n_particles):
for d in range(dim):
r1, r2 = random.random(), random.random()
velocity[i][d] = (w * velocity[i][d]
+ c1 * r1 * (pbest[i][d] - particles[i][d])
+ c2 * r2 * (gbest[d] - particles[i][d]))
particles[i][d] += velocity[i][d]
particles[i][d] = max(bounds[d][0], min(bounds[d][1], particles[i][d]))
val = objective(particles[i])
if val < pbest_val[i]:
pbest[i] = particles[i][:]
pbest_val[i] = val
if val < gbest_val:
gbest = particles[i][:]
gbest_val = val
return gbest, gbest_val
def sphere(x):
return sum(v*v for v in x)
best, val = pso(sphere, [(-10,10)]*3)
print(f"PSO found: {best}, f(x)={val:.6f}")
Chapter Summary
- Understand the core concepts and theory
- Master implementation methods and techniques
- Learn common issues and their solutions
- Apply knowledge to real-world projects
Further Reading
- Official documentation and API references
- Open source projects on GitHub
- Related technical books and courses
- Community discussions and technical blogs
Implementation Examples
Basic Examples
# This section provides a complete implementation example
# to help you apply what you've learned to real projects
Steps
- Initialization: Set up the development environment and required tools
- Data Preparation: Collect and organize the required data
- Core Implementation: Implement the main functionality and logic
- Testing & Validation: Ensure the functionality works correctly
- Optimization: Tune performance and user experience
Common Errors
| Error Type | Possible Cause | Solution | |-----------|---------------|----------| | Compilation Error | Syntax issues | Check code syntax | | Runtime Error | Environment issues | Verify dependencies are installed | | Logic Error | Algorithm issues | Step-by-step debugging and testing | | Performance Issue | Efficiency issues | Use performance analysis tools |
Code Example
# Example code
import sys
def main():
# Main program logic
print("Hello, World!")
if __name__ == "__main__":
main()
Related Resources
- Official documentation
- API reference manuals
- Open source project examples
- Technical community discussions
Particle Swarm Optimization
How PSO Works
Each particle has:
- Position ($x_i$): current candidate solution
- Velocity ($v_i$): direction and speed
- pbest: personal best position
- gbest: global best position (from entire swarm)
Velocity Update
$$v_i(t+1) = w \cdot v_i(t) + c_1 r_1 (pbest_i - x_i) + c_2 r_2 (gbest - x_i)$$
$$x_i(t+1) = x_i(t) + v_i(t+1)$$
| Parameter | Meaning | Typical Value | |-----------|---------|---------------| | $w$ | Inertia weight (exploration) | 0.7 | | $c_1$ | Cognitive coefficient (self) | 1.5 | | $c_2$ | Social coefficient (swarm) | 1.5 | | Swarm size | Number of particles | 30 |
Python Implementation
import random
import math
class Particle:
def __init__(self, dim, bounds):
self.position = [random.uniform(b[0], b[1]) for b in bounds]
self.velocity = [random.uniform(-1, 1) for _ in range(dim)]
self.pbest = self.position.copy()
self.pbest_value = float('inf')
def particle_swarm(cost_func, dim, bounds, swarm_size=30, w=0.7, c1=1.5, c2=1.5, max_iter=100):
swarm = [Particle(dim, bounds) for _ in range(swarm_size)]
gbest = swarm[0].position.copy()
gbest_value = float('inf')
for iteration in range(max_iter):
for particle in swarm:
value = cost_func(particle.position)
if value < particle.pbest_value:
particle.pbest = particle.position.copy()
particle.pbest_value = value
if value < gbest_value:
gbest = particle.position.copy()
gbest_value = value
for particle in swarm:
for i in range(dim):
r1, r2 = random.random(), random.random()
cognitive = c1 * r1 * (particle.pbest[i] - particle.position[i])
social = c2 * r2 * (gbest[i] - particle.position[i])
particle.velocity[i] = w * particle.velocity[i] + cognitive + social
particle.position[i] += particle.velocity[i]
particle.position[i] = max(bounds[i][0], min(bounds[i][1], particle.position[i]))
if iteration % 20 == 0:
print(f"Iter {iteration}: best = {gbest_value:.6f}")
return gbest, gbest_value
# Example: minimize Sphere function
def sphere(x):
return sum(xi**2 for xi in x)
best_pos, best_val = particle_swarm(sphere, dim=3, bounds=[(-10,10)]*3, max_iter=100)
print(f"Best: {best_pos}, Value: {best_val:.6f}")
Differential Evolution
How DE Works
- Mutation: Create donor vector from three random population members
- Crossover: Mix donor with target to create trial vector
- Selection: If trial is better than target, replace it
Mutation Strategies
| Strategy | Formula | |----------|---------| | rand/1 | $v = x_{r1} + F \cdot (x_{r2} - x_{r3})$ | | best/1 | $v = x_{best} + F \cdot (x_{r1} - x_{r2})$ | | current-to-best/1 | $v = x_i + F \cdot (x_{best} - x_i) + F \cdot (x_{r1} - x_{r2})$ |
Summary
PSO and DE are population-based optimization algorithms. PSO is inspired by bird flocking; DE by evolutionary operators.
Key takeaways: | PSO: particles share info via pbest and gbest | | Velocity = inertia ร old + cognitive + social components | | DE: mutation (F), crossover (CR), selection โ three simple operators | | PSO parameters: w (inertia), c1 (self), c2 (swarm) | | DE parameters: F (0.5-1.0), CR (0.7-0.9), population size | | Both handle continuous optimization without gradients | | DE is more robust for multimodal problems | | PSO converges faster but may miss global optimum |
Next Chapter: Hyperparameter Tuning
The next chapter covers hyperparameter tuning with metaheuristics.